2024-09-14T23:57:51.633 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T23:57:51.650 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T23:57:51.723 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7904812' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_packages: - rbd-nbd flavor: default sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 4271 sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 sleep_before_teardown: 0 subset: 97/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMaaqULEJUBl8gf4QfIOoAkM81E2NSuXrn1aVFZDTd/CMyYmw5QazkTn/4bFBur/xubsieQcD3qsxFTFgd0RC/o= smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHP03GaIXMYrkiGBHT2/QmEQ7JGBgKl8K3PITsZt30dxNGeTV8nuSXq9OaRMalF/28Mimtbs1amUL1E6ndFqtzE= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIkrFPLjFX7ySN3YxB2gqiu5JhWWYXuImBe6VXTcw6Obgu19Imj4fTZvpVecUAlX+pxMOxbNazVZyxR0nnu+mUQ= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-14_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T23:57:51.723 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa; will attempt to use it 2024-09-14T23:57:51.724 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa/tasks 2024-09-14T23:57:51.724 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T23:57:51.725 INFO:teuthology.task.internal:Checking packages... 2024-09-14T23:57:51.747 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ecab7be58963a5bd03910cd73eef66405d7fdd50' 2024-09-14T23:57:51.747 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T23:57:51.747 INFO:teuthology.packaging:ref: None 2024-09-14T23:57:51.747 INFO:teuthology.packaging:tag: None 2024-09-14T23:57:51.747 INFO:teuthology.packaging:branch: squid 2024-09-14T23:57:51.747 INFO:teuthology.packaging:sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T23:57:51.747 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-14T23:57:51.942 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-324.gecab7be5 2024-09-14T23:57:51.943 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T23:57:51.950 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T23:57:51.950 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T23:57:52.070 INFO:teuthology.task.internal:Saving configuration 2024-09-14T23:57:52.083 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T23:57:52.107 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T23:57:52.132 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 23:51:45.169261', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMaaqULEJUBl8gf4QfIOoAkM81E2NSuXrn1aVFZDTd/CMyYmw5QazkTn/4bFBur/xubsieQcD3qsxFTFgd0RC/o='} 2024-09-14T23:57:52.152 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 23:51:45.167747', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHP03GaIXMYrkiGBHT2/QmEQ7JGBgKl8K3PITsZt30dxNGeTV8nuSXq9OaRMalF/28Mimtbs1amUL1E6ndFqtzE='} 2024-09-14T23:57:52.176 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 23:51:45.170524', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIkrFPLjFX7ySN3YxB2gqiu5JhWWYXuImBe6VXTcw6Obgu19Imj4fTZvpVecUAlX+pxMOxbNazVZyxR0nnu+mUQ='} 2024-09-14T23:57:52.176 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T23:57:52.183 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-14T23:57:52.183 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-14T23:57:52.183 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['client.0'] 2024-09-14T23:57:52.183 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T23:57:52.266 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fadd08ffac0>, signals=[15]) 2024-09-14T23:57:52.267 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T23:57:52.274 INFO:teuthology.task.internal:Opening connections... 2024-09-14T23:57:52.275 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-09-14T23:57:52.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:57:52.385 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-09-14T23:57:52.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:57:52.462 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-14T23:57:52.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:57:52.537 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T23:57:52.545 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-09-14T23:57:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-09-14T23:57:52.564 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-09-14T23:57:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-09-14T23:57:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-09-14T23:57:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-09-14T23:57:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-09-14T23:57:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-09-14T23:57:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-09-14T23:57:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T23:57:52.624 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-09-14T23:57:52.624 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-09-14T23:57:52.624 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T23:57:52.624 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-09-14T23:57:52.624 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T23:57:52.624 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T23:57:52.624 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T23:57:52.625 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-09-14T23:57:52.651 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-09-14T23:57:52.671 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-09-14T23:57:52.671 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-09-14T23:57:52.729 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-09-14T23:57:52.729 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-09-14T23:57:52.729 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-09-14T23:57:52.729 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-09-14T23:57:52.729 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-09-14T23:57:52.729 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-09-14T23:57:52.729 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T23:57:52.730 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-09-14T23:57:52.730 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-09-14T23:57:52.730 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T23:57:52.730 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-09-14T23:57:52.730 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T23:57:52.730 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T23:57:52.730 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T23:57:52.730 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-09-14T23:57:52.759 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-14T23:57:52.778 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-14T23:57:52.778 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-14T23:57:52.835 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-09-14T23:57:52.835 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-09-14T23:57:52.835 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-09-14T23:57:52.835 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T23:57:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T23:57:52.837 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-14T23:57:52.867 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T23:57:52.877 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T23:57:52.883 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T23:57:52.884 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-09-14T23:57:52.886 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-09-14T23:57:52.888 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-14T23:57:52.904 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T23:57:52.910 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T23:57:52.911 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-09-14T23:57:52.944 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-09-14T23:57:52.947 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-14T23:57:52.977 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T23:57:53.050 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T23:57:53.064 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T23:57:53.064 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T23:57:53.064 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T23:57:53.065 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T23:57:53.065 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T23:57:53.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T23:57:53.082 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-14T23:57:53.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T23:57:53.083 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-14T23:57:53.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T23:57:53.084 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-503.el9.x86_64 2024-09-14T23:57:53.139 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-503.el9.x86_64 2024-09-14T23:57:53.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-503.el9.x86_64 2024-09-14T23:57:53.141 INFO:teuthology.task.kernel:Running kernel on smithi079: 5.14.0-503.el9.x86_64 2024-09-14T23:57:53.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum install -y kernel 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-503.el9.x86_64 2024-09-14T23:57:53.142 INFO:teuthology.task.kernel:Running kernel on smithi027: 5.14.0-503.el9.x86_64 2024-09-14T23:57:53.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2024-09-14T23:57:55.406 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-09-14T23:57:55.775 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.2 MB 00:00 2024-09-14T23:57:57.149 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-14T23:57:58.976 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 20 MB 00:02 2024-09-14T23:57:59.891 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.2 MB 00:04 2024-09-14T23:58:01.475 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-14T23:58:01.892 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-14T23:58:02.975 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-14T23:58:03.887 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-14T23:58:04.934 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-14T23:58:06.284 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-14T23:58:06.418 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-14T23:58:07.347 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-14T23:58:08.110 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-14T23:58:11.341 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T23:58:11.341 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T23:58:11.434 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-14T23:58:11.446 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-14T23:58:11.447 INFO:teuthology.orchestra.run.smithi079.stdout:Install 4 Packages 2024-09-14T23:58:11.448 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-14T23:58:11.448 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 79 M 2024-09-14T23:58:11.448 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 126 M 2024-09-14T23:58:11.448 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-14T23:58:11.544 INFO:teuthology.orchestra.run.smithi079.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 299 kB/s | 23 kB 00:00 2024-09-14T23:58:11.902 INFO:teuthology.orchestra.run.smithi079.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-14T23:58:12.411 INFO:teuthology.orchestra.run.smithi079.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi079.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi079.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-14T23:58:13.414 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-14T23:58:13.738 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-14T23:58:13.738 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-14T23:58:14.536 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-14T23:58:14.711 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-14T23:58:14.711 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-14T23:58:14.767 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T23:58:14.767 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T23:58:14.855 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-14T23:58:14.864 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-09-14T23:58:14.865 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-14T23:58:14.866 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 79 M 2024-09-14T23:58:14.866 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 126 M 2024-09-14T23:58:14.866 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-14T23:58:15.237 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 251 kB/s | 23 kB 00:00 2024-09-14T23:58:15.612 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-14T23:58:15.987 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 38 MB/s | 29 MB 00:00 2024-09-14T23:58:16.296 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 30 MB/s | 35 MB 00:01 2024-09-14T23:58:16.296 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:58:16.296 INFO:teuthology.orchestra.run.smithi092.stdout:Total 55 MB/s | 79 MB 00:01 2024-09-14T23:58:16.660 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T23:58:16.660 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T23:58:16.750 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-14T23:58:16.756 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-14T23:58:16.756 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout:Install 4 Packages 2024-09-14T23:58:16.757 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-14T23:58:16.758 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 79 M 2024-09-14T23:58:16.758 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 126 M 2024-09-14T23:58:16.758 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi027.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 195 kB/s | 23 kB 00:00 2024-09-14T23:58:17.137 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-14T23:58:17.347 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-14T23:58:17.347 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi027.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-09-14T23:58:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:58:17.853 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:58:18.147 INFO:teuthology.orchestra.run.smithi027.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 27 MB/s | 35 MB 00:01 2024-09-14T23:58:18.328 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-14T23:58:18.329 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi027.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 18 MB/s | 29 MB 00:01 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi027.stdout:Total 43 MB/s | 79 MB 00:01 2024-09-14T23:58:19.317 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-14T23:58:19.747 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-14T23:58:19.747 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-14T23:58:20.628 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:58:20.771 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-14T23:58:20.771 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-14T23:58:20.772 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:58:20.783 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-14T23:58:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:58:21.537 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:58:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-14T23:58:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:58:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:58:24.320 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:58:24.458 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:58:25.588 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:58:25.706 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:58:27.051 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:58:27.177 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:58:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:58:29.308 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:58:32.405 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:58:32.435 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:58:34.148 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:58:37.687 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:09.247 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:09.247 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:59:09.247 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:59:09.248 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:59:10.082 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:10.082 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-14T23:59:10.082 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-14T23:59:10.082 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T23:59:10.083 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T23:59:10.083 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T23:59:10.083 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T23:59:10.083 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-14T23:59:10.083 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-14T23:59:10.468 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:59:10.559 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:59:10.559 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:59:10.559 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T23:59:10.559 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T23:59:10.559 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T23:59:10.560 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T23:59:10.560 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi079.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T23:59:10.560 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:59:10.619 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:59:10.619 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:59:10.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo rpm -qi grub2-tools 2024-09-14T23:59:10.658 INFO:teuthology.orchestra.run.smithi079.stdout:Name : grub2-tools 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Epoch : 1 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Version : 2.06 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Release : 82.el9 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: x86_64 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Group : Unspecified 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Size : 8274670 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:License : GPLv3+ 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T23:59:10.659 INFO:teuthology.orchestra.run.smithi079.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:Packager : builder@centos.org 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:Vendor : CentOS 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:Summary : Support tools for GRUB. 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:Description : 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T23:59:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T23:59:10.661 INFO:teuthology.orchestra.run.smithi079.stdout:hardware devices. 2024-09-14T23:59:10.661 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-14T23:59:10.661 INFO:teuthology.orchestra.run.smithi079.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T23:59:10.662 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T23:59:10.662 INFO:teuthology.task.kernel:Updating grub on smithi079 to boot 5.14.0-505.el9.x86_64 2024-09-14T23:59:10.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T23:59:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:Generating grub configuration file ... 2024-09-14T23:59:12.548 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi079.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:12.596 INFO:teuthology.orchestra.run.smithi079.stderr:done 2024-09-14T23:59:12.598 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.EJ7HqiZxhI 2024-09-14T23:59:12.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EJ7HqiZxhI 2024-09-14T23:59:12.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.EJ7HqiZxhI 2024-09-14T23:59:12.819 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.EJ7HqiZxhI is 6KB 2024-09-14T23:59:12.867 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.EJ7HqiZxhI 2024-09-14T23:59:12.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T23:59:12.946 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T23:59:12.946 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T23:59:12.946 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T23:59:12.946 INFO:teuthology.orchestra.run.smithi079.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T23:59:12.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T23:59:12.983 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:12.983 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:59:12.983 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:59:12.983 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:59:13.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo shutdown -r now 2024-09-14T23:59:13.820 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:13.821 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-14T23:59:13.821 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-14T23:59:13.821 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T23:59:13.821 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T23:59:13.821 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T23:59:13.821 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T23:59:13.821 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-14T23:59:13.821 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-14T23:59:14.153 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:59:14.232 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:59:14.232 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T23:59:14.232 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T23:59:14.233 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T23:59:14.233 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-14T23:59:14.233 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T23:59:14.233 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:59:14.291 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:59:14.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2024-09-14T23:59:14.322 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2024-09-14T23:59:14.322 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2024-09-14T23:59:14.322 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2024-09-14T23:59:14.322 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 82.el9 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8274670 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2024-09-14T23:59:14.323 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-14T23:59:14.324 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T23:59:14.325 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T23:59:14.325 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 5.14.0-505.el9.x86_64 2024-09-14T23:59:14.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T23:59:14.945 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-09-14T23:59:16.310 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T23:59:16.338 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-09-14T23:59:16.340 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-14T23:59:16.356 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.PshibP341k 2024-09-14T23:59:16.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PshibP341k 2024-09-14T23:59:16.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.PshibP341k 2024-09-14T23:59:16.578 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.PshibP341k is 6KB 2024-09-14T23:59:16.627 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.PshibP341k 2024-09-14T23:59:16.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T23:59:16.708 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T23:59:16.709 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T23:59:16.709 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T23:59:16.709 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T23:59:16.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T23:59:16.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-09-14T23:59:17.600 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:17.642 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:18.077 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:18.077 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:59:18.077 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:59:18.077 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:59:18.887 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:59:18.887 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-14T23:59:18.887 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-14T23:59:18.887 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T23:59:18.887 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T23:59:18.887 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T23:59:18.888 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T23:59:18.888 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-14T23:59:18.888 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-14T23:59:19.259 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:59:19.345 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:59:19.345 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:59:19.346 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T23:59:19.346 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T23:59:19.346 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T23:59:19.346 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T23:59:19.346 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T23:59:19.346 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:59:19.408 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:59:19.408 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 82.el9 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8274670 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T23:59:19.444 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T23:59:19.444 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 5.14.0-505.el9.x86_64 2024-09-14T23:59:19.444 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T23:59:20.083 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-09-14T23:59:21.387 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T23:59:21.407 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-09-14T23:59:21.410 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-14T23:59:21.430 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.ru26FtgVGf 2024-09-14T23:59:21.430 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ru26FtgVGf 2024-09-14T23:59:21.498 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.ru26FtgVGf 2024-09-14T23:59:21.640 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.ru26FtgVGf is 6KB 2024-09-14T23:59:21.690 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.ru26FtgVGf 2024-09-14T23:59:21.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T23:59:21.770 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T23:59:21.770 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T23:59:21.770 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T23:59:21.770 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T23:59:21.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T23:59:21.921 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-09-14T23:59:43.117 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T23:59:43.118 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-14T23:59:43.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:59:46.861 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-14T23:59:46.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-14T23:59:46.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:59:51.930 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T23:59:51.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-14T23:59:51.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:00:01.597 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-15T00:00:05.181 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-15T00:00:10.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-15T00:00:10.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:00:13.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-15T00:00:14.183 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T00:00:14.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:00:17.245 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-15T00:00:25.665 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-15T00:00:25.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:00:26.433 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-15T00:00:29.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T00:00:29.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:00:35.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T00:00:35.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:00:38.493 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-15T00:00:47.677 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-15T00:00:50.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T00:00:50.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:00:53.565 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-15T00:00:57.505 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-15T00:00:58.046 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-15T00:00:58.047 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-15T00:00:58.047 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-15T00:00:58.077 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-505.el9.x86_64 2024-09-15T00:00:58.078 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T00:00:58.078 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T00:00:58.078 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T00:00:59.079 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-15T00:00:59.080 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T00:00:59.146 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2024-09-15T00:00:59.173 DEBUG:teuthology.parallel:result is None 2024-09-15T00:01:02.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T00:01:02.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:01:03.019 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-15T00:01:03.403 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T00:01:03.403 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-15T00:01:03.404 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-15T00:01:03.417 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-505.el9.x86_64 2024-09-15T00:01:03.418 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T00:01:03.418 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T00:01:03.418 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T00:01:04.418 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-15T00:01:04.419 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T00:01:04.497 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-09-15T00:01:04.517 DEBUG:teuthology.parallel:result is None 2024-09-15T00:01:08.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T00:01:08.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:01:27.101 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-15T00:01:45.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T00:01:45.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:01:45.471 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-15T00:01:45.823 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T00:01:45.824 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-15T00:01:45.824 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-15T00:01:45.840 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-505.el9.x86_64 2024-09-15T00:01:45.840 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T00:01:45.840 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T00:01:45.841 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T00:01:46.841 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T00:01:46.842 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T00:01:46.910 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2024-09-15T00:01:46.933 DEBUG:teuthology.parallel:result is None 2024-09-15T00:01:46.933 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T00:01:46.954 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T00:01:46.955 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T00:01:46.957 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T00:01:46.960 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T00:01:46.979 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T00:01:46.995 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T00:01:47.034 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T00:01:47.035 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T00:01:47.037 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T00:01:47.040 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T00:01:47.155 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T00:01:47.162 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T00:01:47.163 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T00:01:47.165 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T00:01:47.167 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T00:01:47.196 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:01:47.197 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:01:47.208 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:01:47.210 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:01:47.213 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:01:47.225 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T00:01:47.227 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T00:01:47.235 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T00:01:47.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T00:01:47.252 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T00:01:47.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T00:01:47.283 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T00:01:47.291 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T00:01:47.291 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T00:01:47.320 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T00:01:47.322 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T00:01:47.340 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:01:47.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:01:47.495 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:01:47.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:01:47.579 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:01:47.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T00:01:47.644 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:01:47.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:01:47.768 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:01:47.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:01:47.854 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:01:47.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T00:01:47.920 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:01:47.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T00:01:48.045 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:01:48.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T00:01:48.131 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T00:01:48.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T00:01:48.198 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-09-15T00:01:48.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-09-15T00:01:48.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-15T00:01:48.237 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:01:48.238 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:01:48.290 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:01:48.788 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T00:01:48.795 INFO:teuthology.task.internal:Starting timer... 2024-09-15T00:01:48.795 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T00:01:48.810 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T00:01:48.821 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-15T00:01:48.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-09-15T00:01:48.909 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-09-15T00:01:48.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2024-09-15T00:01:48.984 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2024-09-15T00:01:48.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-09-15T00:01:49.140 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-09-15T00:01:49.142 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T00:01:49.142 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-09-15T00:01:49.169 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-09-15T00:01:49.170 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2024-09-15T00:01:49.195 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2024-09-15T00:01:49.195 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-09-15T00:01:49.224 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-09-15T00:01:49.224 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi079.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive'} 2024-09-15T00:01:49.224 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:01:49.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:01:49.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:01:49.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:01:49.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:01:49.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:01:49.312 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T00:01:49.312 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-09-15T00:01:49.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2024-09-15T00:01:49.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-09-15T00:01:49.411 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T00:01:49.497 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T00:01:49.508 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T00:01:49.510 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi079.front.sepia.ceph.com,smithi092.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T00:05:36.430 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com')] 2024-09-15T00:05:36.431 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T00:05:36.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:05:36.515 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-15T00:05:36.597 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T00:05:36.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-15T00:05:36.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:05:36.675 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-15T00:05:36.752 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-15T00:05:36.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T00:05:36.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T00:05:36.826 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-15T00:05:36.905 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T00:05:36.906 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T00:05:36.915 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T00:05:36.916 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T00:05:36.916 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:05:36.919 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T00:05:36.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:05:36.921 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T00:05:36.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:05:36.947 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T00:05:36.952 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T00:05:36.962 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T00:05:36.969 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T00:05:36.973 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T00:05:36.986 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2024-09-15T00:05:36.988 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T00:05:36.997 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2024-09-15T00:05:36.997 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-09-15T00:05:37.009 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-09-15T00:05:37.010 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T00:05:37.012 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-09-15T00:05:37.023 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-09-15T00:05:37.024 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T00:05:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T00:05:37.038 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T00:05:37.040 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T00:05:37.052 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T00:05:37.080 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:05:37.083 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:05:37.083 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-09-15T00:05:37.084 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.084 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.084 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.084 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.099 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:05:37.102 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:05:37.102 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-15T00:05:37.102 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.102 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.102 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.103 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.112 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:05:37.115 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:05:37.116 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-15T00:05:37.116 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.116 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.116 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.116 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:05:37.117 INFO:teuthology.run_tasks:Running task install... 2024-09-15T00:05:37.124 DEBUG:teuthology.task.install:project ceph 2024-09-15T00:05:37.124 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-15T00:05:37.125 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'} 2024-09-15T00:05:37.125 INFO:teuthology.task.install:Using flavor: default 2024-09-15T00:05:37.130 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T00:05:37.130 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-15T00:05:37.131 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'tag': None, 'wait_for_package': False} 2024-09-15T00:05:37.131 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:05:37.133 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'tag': None, 'wait_for_package': False} 2024-09-15T00:05:37.134 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:05:37.134 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'tag': None, 'wait_for_package': False} 2024-09-15T00:05:37.135 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:05:37.272 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/ 2024-09-15T00:05:37.272 INFO:teuthology.task.install.rpm:Package version is 19.1.1-324.gecab7be5 2024-09-15T00:05:37.287 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/ 2024-09-15T00:05:37.287 INFO:teuthology.task.install.rpm:Package version is 19.1.1-324.gecab7be5 2024-09-15T00:05:37.290 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/ 2024-09-15T00:05:37.290 INFO:teuthology.task.install.rpm:Package version is 19.1.1-324.gecab7be5 2024-09-15T00:05:37.383 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T00:05:37.384 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:05:37.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T00:05:37.397 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T00:05:37.398 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T00:05:37.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T00:05:37.408 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T00:05:37.408 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:05:37.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T00:05:37.410 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-15T00:05:37.411 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ecab7be58963a5bd03910cd73eef66405d7fdd50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T00:05:37.424 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-15T00:05:37.424 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ecab7be58963a5bd03910cd73eef66405d7fdd50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T00:05:37.434 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-15T00:05:37.434 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ecab7be58963a5bd03910cd73eef66405d7fdd50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T00:05:37.478 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T00:05:37.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T00:05:37.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T00:05:37.555 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T00:05:37.565 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T00:05:37.575 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T00:05:37.620 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-09-15T00:05:37.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-09-15T00:05:37.629 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-09-15T00:05:37.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-09-15T00:05:37.639 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2024-09-15T00:05:37.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2024-09-15T00:05:37.852 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-09-15T00:05:37.866 INFO:teuthology.orchestra.run.smithi079.stdout:45 files removed 2024-09-15T00:05:37.867 INFO:teuthology.orchestra.run.smithi027.stdout:45 files removed 2024-09-15T00:05:37.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw 2024-09-15T00:05:37.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw 2024-09-15T00:05:37.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw 2024-09-15T00:05:38.436 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-15T00:05:38.445 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-15T00:05:38.450 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 325 kB/s | 84 kB 00:00 2024-09-15T00:05:38.736 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-15T00:05:38.738 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-15T00:05:38.743 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-15T00:05:39.002 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 8.6 kB/s | 1.9 kB 00:00 2024-09-15T00:05:39.012 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T00:05:39.013 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T00:05:39.577 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-15T00:05:40.854 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-15T00:05:40.855 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-15T00:05:42.579 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-15T00:05:42.649 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-15T00:05:46.875 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-15T00:05:48.470 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-15T00:05:48.534 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-15T00:05:48.580 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-15T00:05:49.413 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-15T00:05:50.289 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-15T00:05:51.070 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-15T00:05:52.862 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-15T00:05:54.580 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-15T00:05:54.877 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-15T00:05:55.617 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-15T00:05:56.559 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-15T00:05:57.253 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:05:57.270 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:05:57.270 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:05:57.270 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:05:57.270 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:05:57.270 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 11 M 2024-09-15T00:05:57.270 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-09-15T00:05:57.270 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.4 M 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.2 M 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.4 M 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 21 M 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:19.1.1-324.gecab7be5.el9 ceph 25 k 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 752 k 2024-09-15T00:05:57.271 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T00:05:57.272 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T00:05:57.272 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T00:05:57.272 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 507 k 2024-09-15T00:05:57.272 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T00:05:57.272 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.3 M 2024-09-15T00:05:57.272 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T00:05:57.272 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T00:05:57.272 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T00:05:57.272 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 45 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 136 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:19.1.1-324.gecab7be5.el9 ceph 162 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:19.1.1-324.gecab7be5.el9 ceph 321 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 301 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 99 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T00:05:57.273 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T00:05:57.274 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2024-09-15T00:05:57.274 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T00:05:57.274 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:05:57.274 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:05:57.274 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:05:57.274 INFO:teuthology.orchestra.run.smithi079.stdout:Install 30 Packages 2024-09-15T00:05:57.274 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 2 Packages 2024-09-15T00:05:57.274 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:05:57.275 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 60 M 2024-09-15T00:05:57.275 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:05:58.212 INFO:teuthology.orchestra.run.smithi079.stdout:(1/32): ceph-base-19.1.1-324.gecab7be5.el9.x86_ 11 MB/s | 5.4 MB 00:00 2024-09-15T00:05:58.270 INFO:teuthology.orchestra.run.smithi079.stdout:(2/32): ceph-selinux-19.1.1-324.gecab7be5.el9.x 432 kB/s | 25 kB 00:00 2024-09-15T00:05:58.371 INFO:teuthology.orchestra.run.smithi079.stdout:(3/32): libcephfs2-19.1.1-324.gecab7be5.el9.x86 7.3 MB/s | 752 kB 00:00 2024-09-15T00:05:58.463 INFO:teuthology.orchestra.run.smithi079.stdout:(4/32): libradosstriper1-19.1.1-324.gecab7be5.e 5.4 MB/s | 507 kB 00:00 2024-09-15T00:05:58.688 INFO:teuthology.orchestra.run.smithi079.stdout:(5/32): ceph-radosgw-19.1.1-324.gecab7be5.el9.x 11 MB/s | 11 MB 00:00 2024-09-15T00:05:58.905 INFO:teuthology.orchestra.run.smithi079.stdout:(6/32): ceph-common-19.1.1-324.gecab7be5.el9.x8 18 MB/s | 21 MB 00:01 2024-09-15T00:05:58.918 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:05:58.928 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:05:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:05:58.928 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:05:58.928 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:05:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 11 M 2024-09-15T00:05:58.928 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-09-15T00:05:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.4 M 2024-09-15T00:05:58.928 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.2 M 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.4 M 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 21 M 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.1.1-324.gecab7be5.el9 ceph 25 k 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 752 k 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T00:05:58.929 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 507 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.3 M 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 45 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 136 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.1.1-324.gecab7be5.el9 ceph 162 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.1.1-324.gecab7be5.el9 ceph 321 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 301 k 2024-09-15T00:05:58.930 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 99 k 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout:Install 30 Packages 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2024-09-15T00:05:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:05:58.932 INFO:teuthology.orchestra.run.smithi079.stdout:(7/32): python3-ceph-argparse-19.1.1-324.gecab7 186 kB/s | 45 kB 00:00 2024-09-15T00:05:58.932 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 60 M 2024-09-15T00:05:58.932 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:05:58.956 INFO:teuthology.orchestra.run.smithi079.stdout:(8/32): python3-ceph-common-19.1.1-324.gecab7be 2.7 MB/s | 136 kB 00:00 2024-09-15T00:05:58.981 INFO:teuthology.orchestra.run.smithi079.stdout:(9/32): python3-cephfs-19.1.1-324.gecab7be5.el9 3.1 MB/s | 162 kB 00:00 2024-09-15T00:05:59.007 INFO:teuthology.orchestra.run.smithi079.stdout:(10/32): python3-rados-19.1.1-324.gecab7be5.el9 6.3 MB/s | 321 kB 00:00 2024-09-15T00:05:59.032 INFO:teuthology.orchestra.run.smithi079.stdout:(11/32): python3-rbd-19.1.1-324.gecab7be5.el9.x 5.9 MB/s | 301 kB 00:00 2024-09-15T00:05:59.057 INFO:teuthology.orchestra.run.smithi079.stdout:(12/32): python3-rgw-19.1.1-324.gecab7be5.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-15T00:05:59.133 INFO:teuthology.orchestra.run.smithi079.stdout:(13/32): librgw2-19.1.1-324.gecab7be5.el9.x86_6 8.0 MB/s | 5.3 MB 00:00 2024-09-15T00:05:59.466 INFO:teuthology.orchestra.run.smithi079.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 136 kB/s | 45 kB 00:00 2024-09-15T00:05:59.492 INFO:teuthology.orchestra.run.smithi079.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 233 kB/s | 107 kB 00:00 2024-09-15T00:05:59.517 INFO:teuthology.orchestra.run.smithi079.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 349 kB/s | 160 kB 00:00 2024-09-15T00:05:59.859 INFO:teuthology.orchestra.run.smithi079.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 124 kB/s | 42 kB 00:00 2024-09-15T00:05:59.992 INFO:teuthology.orchestra.run.smithi079.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 584 kB/s | 292 kB 00:00 2024-09-15T00:05:59.993 INFO:teuthology.orchestra.run.smithi092.stdout:(1/32): ceph-base-19.1.1-324.gecab7be5.el9.x86_ 11 MB/s | 5.4 MB 00:00 2024-09-15T00:06:00.060 INFO:teuthology.orchestra.run.smithi092.stdout:(2/32): ceph-selinux-19.1.1-324.gecab7be5.el9.x 378 kB/s | 25 kB 00:00 2024-09-15T00:06:00.126 INFO:teuthology.orchestra.run.smithi079.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.0 MB/s | 662 kB 00:00 2024-09-15T00:06:00.160 INFO:teuthology.orchestra.run.smithi092.stdout:(3/32): libcephfs2-19.1.1-324.gecab7be5.el9.x86 7.4 MB/s | 752 kB 00:00 2024-09-15T00:06:00.251 INFO:teuthology.orchestra.run.smithi079.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 139 kB/s | 36 kB 00:00 2024-09-15T00:06:00.252 INFO:teuthology.orchestra.run.smithi092.stdout:(4/32): libradosstriper1-19.1.1-324.gecab7be5.e 5.4 MB/s | 507 kB 00:00 2024-09-15T00:06:00.310 INFO:teuthology.orchestra.run.smithi079.stdout:(21/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 5.2 MB/s | 308 kB 00:00 2024-09-15T00:06:00.352 INFO:teuthology.orchestra.run.smithi079.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 448 kB/s | 221 kB 00:00 2024-09-15T00:06:00.402 INFO:teuthology.orchestra.run.smithi079.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-15T00:06:00.436 INFO:teuthology.orchestra.run.smithi079.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-15T00:06:00.461 INFO:teuthology.orchestra.run.smithi092.stdout:(5/32): ceph-radosgw-19.1.1-324.gecab7be5.el9.x 11 MB/s | 11 MB 00:00 2024-09-15T00:06:00.461 INFO:teuthology.orchestra.run.smithi079.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T00:06:00.487 INFO:teuthology.orchestra.run.smithi079.stdout:(26/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 132 kB/s | 47 kB 00:00 2024-09-15T00:06:00.545 INFO:teuthology.orchestra.run.smithi079.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T00:06:00.579 INFO:teuthology.orchestra.run.smithi079.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-15T00:06:00.669 INFO:teuthology.orchestra.run.smithi092.stdout:(6/32): ceph-common-19.1.1-324.gecab7be5.el9.x8 18 MB/s | 21 MB 00:01 2024-09-15T00:06:00.671 INFO:teuthology.orchestra.run.smithi079.stdout:(29/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-15T00:06:00.694 INFO:teuthology.orchestra.run.smithi092.stdout:(7/32): python3-ceph-argparse-19.1.1-324.gecab7 193 kB/s | 45 kB 00:00 2024-09-15T00:06:00.713 INFO:teuthology.orchestra.run.smithi079.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 9.4 MB/s | 1.6 MB 00:00 2024-09-15T00:06:00.720 INFO:teuthology.orchestra.run.smithi092.stdout:(8/32): python3-ceph-common-19.1.1-324.gecab7be 2.7 MB/s | 136 kB 00:00 2024-09-15T00:06:00.745 INFO:teuthology.orchestra.run.smithi092.stdout:(9/32): python3-cephfs-19.1.1-324.gecab7be5.el9 3.1 MB/s | 162 kB 00:00 2024-09-15T00:06:00.820 INFO:teuthology.orchestra.run.smithi092.stdout:(10/32): librgw2-19.1.1-324.gecab7be5.el9.x86_6 9.4 MB/s | 5.3 MB 00:00 2024-09-15T00:06:00.845 INFO:teuthology.orchestra.run.smithi092.stdout:(11/32): python3-rados-19.1.1-324.gecab7be5.el9 2.5 MB/s | 321 kB 00:00 2024-09-15T00:06:00.855 INFO:teuthology.orchestra.run.smithi079.stdout:(31/32): librados2-19.1.1-324.gecab7be5.el9.x86 12 MB/s | 3.4 MB 00:00 2024-09-15T00:06:00.888 INFO:teuthology.orchestra.run.smithi092.stdout:(12/32): python3-rbd-19.1.1-324.gecab7be5.el9.x 2.1 MB/s | 301 kB 00:00 2024-09-15T00:06:00.913 INFO:teuthology.orchestra.run.smithi092.stdout:(13/32): python3-rgw-19.1.1-324.gecab7be5.el9.x 1.1 MB/s | 99 kB 00:00 2024-09-15T00:06:00.980 INFO:teuthology.orchestra.run.smithi079.stdout:(32/32): librbd1-19.1.1-324.gecab7be5.el9.x86_6 10 MB/s | 3.2 MB 00:00 2024-09-15T00:06:00.985 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:00.985 INFO:teuthology.orchestra.run.smithi079.stdout:Total 16 MB/s | 60 MB 00:03 2024-09-15T00:06:01.021 INFO:teuthology.orchestra.run.smithi092.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 420 kB/s | 45 kB 00:00 2024-09-15T00:06:01.046 INFO:teuthology.orchestra.run.smithi092.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 532 kB/s | 107 kB 00:00 2024-09-15T00:06:01.072 INFO:teuthology.orchestra.run.smithi092.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-15T00:06:01.085 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:06:01.121 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:06:01.121 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:06:01.122 INFO:teuthology.orchestra.run.smithi092.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 846 kB/s | 42 kB 00:00 2024-09-15T00:06:01.147 INFO:teuthology.orchestra.run.smithi092.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-15T00:06:01.173 INFO:teuthology.orchestra.run.smithi092.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-15T00:06:01.198 INFO:teuthology.orchestra.run.smithi092.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 710 kB/s | 36 kB 00:00 2024-09-15T00:06:01.223 INFO:teuthology.orchestra.run.smithi092.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-15T00:06:01.249 INFO:teuthology.orchestra.run.smithi092.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-15T00:06:01.281 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-15T00:06:01.307 INFO:teuthology.orchestra.run.smithi092.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-15T00:06:01.324 INFO:teuthology.orchestra.run.smithi092.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2024-09-15T00:06:01.341 INFO:teuthology.orchestra.run.smithi092.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 334 kB/s | 26 kB 00:00 2024-09-15T00:06:01.367 INFO:teuthology.orchestra.run.smithi092.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-15T00:06:01.408 INFO:teuthology.orchestra.run.smithi092.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-15T00:06:01.442 INFO:teuthology.orchestra.run.smithi092.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-15T00:06:01.509 INFO:teuthology.orchestra.run.smithi092.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-15T00:06:01.593 INFO:teuthology.orchestra.run.smithi092.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-15T00:06:01.599 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:06:01.599 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:06:01.659 INFO:teuthology.orchestra.run.smithi092.stdout:(31/32): librados2-19.1.1-324.gecab7be5.el9.x86 16 MB/s | 3.4 MB 00:00 2024-09-15T00:06:01.751 INFO:teuthology.orchestra.run.smithi092.stdout:(32/32): librbd1-19.1.1-324.gecab7be5.el9.x86_6 13 MB/s | 3.2 MB 00:00 2024-09-15T00:06:01.756 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:01.756 INFO:teuthology.orchestra.run.smithi092.stdout:Total 21 MB/s | 60 MB 00:02 2024-09-15T00:06:01.857 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:06:01.892 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:06:01.892 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:06:02.364 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:06:02.364 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:06:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:06:03.210 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-15T00:06:03.252 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-15T00:06:03.286 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-15T00:06:03.309 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:06:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:06:03.528 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:06:03.581 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:06:03.803 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:06:03.821 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:06:03.822 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:03.823 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:06:03.823 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:03.823 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:06:03.823 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 11 M 2024-09-15T00:06:03.823 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-09-15T00:06:03.823 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.4 M 2024-09-15T00:06:03.823 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.2 M 2024-09-15T00:06:03.823 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-15T00:06:03.823 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.4 M 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 21 M 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.1.1-324.gecab7be5.el9 ceph 25 k 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 752 k 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T00:06:03.824 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 507 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.3 M 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 45 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 136 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.1.1-324.gecab7be5.el9 ceph 162 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.1.1-324.gecab7be5.el9 ceph 321 k 2024-09-15T00:06:03.825 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 301 k 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 99 k 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:03.826 INFO:teuthology.orchestra.run.smithi027.stdout:Install 30 Packages 2024-09-15T00:06:03.827 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 2 Packages 2024-09-15T00:06:03.827 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:03.827 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 60 M 2024-09-15T00:06:03.827 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:06:03.898 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-15T00:06:03.931 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-15T00:06:03.956 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-15T00:06:03.959 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-15T00:06:03.979 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:06:04.004 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:06:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:06:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:06:04.198 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:06:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 8/34 2024-09-15T00:06:04.223 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:06:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:06:04.286 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:06:04.314 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-15T00:06:04.363 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 11/34 2024-09-15T00:06:04.435 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:06:04.459 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:06:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:06:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-15T00:06:04.624 INFO:teuthology.orchestra.run.smithi027.stdout:(1/32): ceph-base-19.1.1-324.gecab7be5.el9.x86_ 11 MB/s | 5.4 MB 00:00 2024-09-15T00:06:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-15T00:06:04.664 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-15T00:06:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:(2/32): ceph-selinux-19.1.1-324.gecab7be5.el9.x 504 kB/s | 25 kB 00:00 2024-09-15T00:06:04.691 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-15T00:06:04.708 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:06:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-15T00:06:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:(3/32): libcephfs2-19.1.1-324.gecab7be5.el9.x86 7.3 MB/s | 752 kB 00:00 2024-09-15T00:06:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-15T00:06:04.800 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:06:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-15T00:06:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-15T00:06:04.875 INFO:teuthology.orchestra.run.smithi027.stdout:(4/32): libradosstriper1-19.1.1-324.gecab7be5.e 5.0 MB/s | 507 kB 00:00 2024-09-15T00:06:04.890 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 8/34 2024-09-15T00:06:04.917 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:06:04.980 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:06:05.017 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-15T00:06:05.067 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 11/34 2024-09-15T00:06:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:06:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-15T00:06:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:06:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:06:05.209 INFO:teuthology.orchestra.run.smithi027.stdout:(5/32): ceph-common-19.1.1-324.gecab7be5.el9.x8 20 MB/s | 21 MB 00:01 2024-09-15T00:06:05.217 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:06:05.251 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-15T00:06:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:06:05.326 INFO:teuthology.orchestra.run.smithi027.stdout:(6/32): ceph-radosgw-19.1.1-324.gecab7be5.el9.x 8.9 MB/s | 11 MB 00:01 2024-09-15T00:06:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 23/34 2024-09-15T00:06:05.351 INFO:teuthology.orchestra.run.smithi027.stdout:(7/32): python3-ceph-argparse-19.1.1-324.gecab7 318 kB/s | 45 kB 00:00 2024-09-15T00:06:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-15T00:06:05.375 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-15T00:06:05.376 INFO:teuthology.orchestra.run.smithi027.stdout:(8/32): python3-ceph-common-19.1.1-324.gecab7be 2.7 MB/s | 136 kB 00:00 2024-09-15T00:06:05.401 INFO:teuthology.orchestra.run.smithi027.stdout:(9/32): python3-cephfs-19.1.1-324.gecab7be5.el9 3.2 MB/s | 162 kB 00:00 2024-09-15T00:06:05.418 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-15T00:06:05.426 INFO:teuthology.orchestra.run.smithi027.stdout:(10/32): python3-rados-19.1.1-324.gecab7be5.el9 6.3 MB/s | 321 kB 00:00 2024-09-15T00:06:05.452 INFO:teuthology.orchestra.run.smithi027.stdout:(11/32): python3-rbd-19.1.1-324.gecab7be5.el9.x 5.8 MB/s | 301 kB 00:00 2024-09-15T00:06:05.469 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-15T00:06:05.477 INFO:teuthology.orchestra.run.smithi027.stdout:(12/32): python3-rgw-19.1.1-324.gecab7be5.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-15T00:06:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-15T00:06:05.530 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-15T00:06:05.561 INFO:teuthology.orchestra.run.smithi027.stdout:(13/32): librgw2-19.1.1-324.gecab7be5.el9.x86_6 7.8 MB/s | 5.3 MB 00:00 2024-09-15T00:06:05.563 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-15T00:06:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-15T00:06:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:06:05.880 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-15T00:06:05.925 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:06:05.947 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:06:06.021 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:06:06.025 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 27/34 2024-09-15T00:06:06.054 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 28/34 2024-09-15T00:06:06.054 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 23/34 2024-09-15T00:06:06.090 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-15T00:06:06.136 INFO:teuthology.orchestra.run.smithi027.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 79 kB/s | 45 kB 00:00 2024-09-15T00:06:06.153 INFO:teuthology.orchestra.run.smithi027.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 152 kB/s | 107 kB 00:00 2024-09-15T00:06:06.203 INFO:teuthology.orchestra.run.smithi027.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 221 kB/s | 160 kB 00:00 2024-09-15T00:06:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-15T00:06:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:06:06.665 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:06:06.743 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 27/34 2024-09-15T00:06:06.771 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 28/34 2024-09-15T00:06:06.795 INFO:teuthology.orchestra.run.smithi027.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 71 kB/s | 42 kB 00:00 2024-09-15T00:06:06.929 INFO:teuthology.orchestra.run.smithi027.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 377 kB/s | 292 kB 00:00 2024-09-15T00:06:07.062 INFO:teuthology.orchestra.run.smithi027.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 716 kB/s | 662 kB 00:00 2024-09-15T00:06:07.479 INFO:teuthology.orchestra.run.smithi027.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 65 kB/s | 36 kB 00:00 2024-09-15T00:06:07.529 INFO:teuthology.orchestra.run.smithi027.stdout:(21/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.1 MB/s | 308 kB 00:00 2024-09-15T00:06:07.555 INFO:teuthology.orchestra.run.smithi027.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 291 kB/s | 221 kB 00:00 2024-09-15T00:06:07.605 INFO:teuthology.orchestra.run.smithi027.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-15T00:06:07.630 INFO:teuthology.orchestra.run.smithi027.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-15T00:06:07.656 INFO:teuthology.orchestra.run.smithi027.stdout:(25/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 80 kB/s | 47 kB 00:00 2024-09-15T00:06:07.680 INFO:teuthology.orchestra.run.smithi027.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-15T00:06:07.723 INFO:teuthology.orchestra.run.smithi027.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.5 MB/s | 191 kB 00:00 2024-09-15T00:06:07.798 INFO:teuthology.orchestra.run.smithi027.stdout:(28/32): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-15T00:06:07.840 INFO:teuthology.orchestra.run.smithi027.stdout:(29/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-09-15T00:06:07.882 INFO:teuthology.orchestra.run.smithi027.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-15T00:06:08.040 INFO:teuthology.orchestra.run.smithi027.stdout:(31/32): librados2-19.1.1-324.gecab7be5.el9.x86 14 MB/s | 3.4 MB 00:00 2024-09-15T00:06:08.157 INFO:teuthology.orchestra.run.smithi027.stdout:(32/32): librbd1-19.1.1-324.gecab7be5.el9.x86_6 10 MB/s | 3.2 MB 00:00 2024-09-15T00:06:08.163 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:08.164 INFO:teuthology.orchestra.run.smithi027.stdout:Total 14 MB/s | 60 MB 00:04 2024-09-15T00:06:08.270 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:06:08.311 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:06:08.311 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:06:08.598 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:06:08.634 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:06:08.814 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:06:08.814 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:06:09.097 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:06:09.142 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:06:09.211 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:06:09.247 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:06:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:06:09.347 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T00:06:09.347 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T00:06:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:06:09.705 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:06:09.745 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:06:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:06:09.828 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T00:06:09.828 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T00:06:09.828 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:06:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:06:10.417 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-15T00:06:10.446 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-15T00:06:10.471 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-15T00:06:10.494 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:06:10.515 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-15T00:06:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:06:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-15T00:06:11.115 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-15T00:06:11.222 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:06:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:06:11.406 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 8/34 2024-09-15T00:06:11.432 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:06:11.497 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-15T00:06:11.538 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-15T00:06:11.600 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 11/34 2024-09-15T00:06:11.681 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:06:11.709 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:06:11.791 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:06:11.833 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-15T00:06:11.952 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-15T00:06:11.984 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-15T00:06:12.029 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-15T00:06:12.065 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-15T00:06:12.095 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-15T00:06:12.129 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-15T00:06:12.464 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-15T00:06:12.507 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:06:12.598 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-15T00:06:12.632 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 23/34 2024-09-15T00:06:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-15T00:06:13.180 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-15T00:06:13.209 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:06:13.281 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-15T00:06:13.361 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 27/34 2024-09-15T00:06:13.395 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 28/34 2024-09-15T00:06:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:06:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:06:16.435 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-15T00:06:16.496 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:06:16.588 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-15T00:06:16.588 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T00:06:16.588 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T00:06:16.588 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:16.622 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:06:19.607 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:06:19.607 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2024-09-15T00:06:19.607 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2024-09-15T00:06:19.607 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2024-09-15T00:06:19.608 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2024-09-15T00:06:19.608 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2024-09-15T00:06:19.608 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /root 2024-09-15T00:06:19.608 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2024-09-15T00:06:19.608 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:19.637 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:06:19.637 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-09-15T00:06:19.637 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-09-15T00:06:19.637 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-09-15T00:06:19.638 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-09-15T00:06:19.638 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-09-15T00:06:19.638 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-09-15T00:06:19.638 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-09-15T00:06:19.638 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:20.558 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:06:20.586 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:06:20.586 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:20.586 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T00:06:20.586 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:06:20.586 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:06:20.586 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:20.588 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:06:20.644 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:06:20.670 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:06:20.670 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:20.670 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T00:06:20.670 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:06:20.670 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:06:20.672 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:20.672 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:06:20.678 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:06:20.678 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:06:20.756 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:06:20.756 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:06:21.862 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:06:21.862 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 1/34 2024-09-15T00:06:21.862 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2/34 2024-09-15T00:06:21.862 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 3/34 2024-09-15T00:06:21.862 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 4/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 5/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 6/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 8/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 9/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 10/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 11/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-15T00:06:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-15T00:06:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-15T00:06:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-15T00:06:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-15T00:06:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-15T00:06:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-15T00:06:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-15T00:06:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-15T00:06:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-15T00:06:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-15T00:06:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-15T00:06:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-15T00:06:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-15T00:06:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-15T00:06:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:06:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-15T00:06:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 33/34 2024-09-15T00:06:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:06:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 1/34 2024-09-15T00:06:21.917 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2/34 2024-09-15T00:06:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 3/34 2024-09-15T00:06:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 4/34 2024-09-15T00:06:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 5/34 2024-09-15T00:06:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 6/34 2024-09-15T00:06:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:06:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 8/34 2024-09-15T00:06:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 9/34 2024-09-15T00:06:21.918 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 10/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 11/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-15T00:06:21.919 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-15T00:06:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-15T00:06:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-15T00:06:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-15T00:06:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-15T00:06:21.920 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-15T00:06:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-15T00:06:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-15T00:06:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:06:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-15T00:06:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 33/34 2024-09-15T00:06:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:06:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:23.041 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-09-15T00:06:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.042 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:06:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T00:06:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T00:06:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T00:06:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T00:06:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T00:06:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T00:06:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T00:06:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T00:06:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T00:06:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T00:06:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T00:06:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T00:06:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T00:06:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T00:06:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T00:06:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T00:06:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T00:06:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:23.047 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:06:23.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.133 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T00:06:23.134 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T00:06:23.135 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T00:06:23.136 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:23.136 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:06:23.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-test 2024-09-15T00:06:23.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-test 2024-09-15T00:06:24.014 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:30 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:06:24.047 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:28 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:06:24.145 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:06:24.145 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:06:24.145 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:24.145 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:19.1.1-324.gecab7be5.el9 ceph 49 M 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:19.1.1-324.gecab7be5.el9 ceph 163 k 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout:Install 4 Packages 2024-09-15T00:06:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:24.147 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 49 M 2024-09-15T00:06:24.147 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 209 M 2024-09-15T00:06:24.147 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:06:24.179 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:06:24.180 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:24.180 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:06:24.180 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:24.180 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:06:24.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.1.1-324.gecab7be5.el9 ceph 49 M 2024-09-15T00:06:24.180 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-15T00:06:24.180 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.1.1-324.gecab7be5.el9 ceph 163 k 2024-09-15T00:06:24.180 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T00:06:24.180 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T00:06:24.181 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:24.181 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:06:24.181 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:24.181 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-09-15T00:06:24.181 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:24.181 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 49 M 2024-09-15T00:06:24.181 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 209 M 2024-09-15T00:06:24.182 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:06:24.380 INFO:teuthology.orchestra.run.smithi079.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-15T00:06:24.422 INFO:teuthology.orchestra.run.smithi079.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-15T00:06:24.518 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-15T00:06:24.568 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T00:06:24.622 INFO:teuthology.orchestra.run.smithi079.stdout:(3/4): libcephsqlite-19.1.1-324.gecab7be5.el9.x 366 kB/s | 163 kB 00:00 2024-09-15T00:06:24.751 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): libcephsqlite-19.1.1-324.gecab7be5.el9.x 347 kB/s | 163 kB 00:00 2024-09-15T00:06:27.172 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:06:27.172 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-09-15T00:06:27.172 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-09-15T00:06:27.172 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-09-15T00:06:27.172 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-09-15T00:06:27.172 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-09-15T00:06:27.172 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-09-15T00:06:27.172 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-09-15T00:06:27.173 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:27.384 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): ceph-test-19.1.1-324.gecab7be5.el9.x86_6 16 MB/s | 49 MB 00:03 2024-09-15T00:06:27.386 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:27.387 INFO:teuthology.orchestra.run.smithi092.stdout:Total 15 MB/s | 49 MB 00:03 2024-09-15T00:06:27.413 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:06:27.434 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:06:27.434 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:06:27.580 INFO:teuthology.orchestra.run.smithi079.stdout:(4/4): ceph-test-19.1.1-324.gecab7be5.el9.x86_6 14 MB/s | 49 MB 00:03 2024-09-15T00:06:27.582 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:27.583 INFO:teuthology.orchestra.run.smithi079.stdout:Total 14 MB/s | 49 MB 00:03 2024-09-15T00:06:27.610 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:06:27.630 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:06:27.630 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:06:27.804 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:06:27.805 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:06:28.001 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:06:28.001 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:06:28.179 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:06:28.207 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-15T00:06:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T00:06:28.208 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:06:28.208 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T00:06:28.208 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:28.209 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:06:28.306 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-15T00:06:28.307 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:06:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:06:28.410 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T00:06:28.450 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T00:06:28.478 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:06:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:06:28.606 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T00:06:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T00:06:28.691 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:06:29.551 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:06:29.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 1/34 2024-09-15T00:06:29.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2/34 2024-09-15T00:06:29.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 3/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 4/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 5/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 6/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 8/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 9/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 10/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 11/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-15T00:06:29.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-15T00:06:29.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-15T00:06:29.554 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-15T00:06:29.554 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-15T00:06:29.554 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-15T00:06:29.555 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-15T00:06:29.555 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 33/34 2024-09-15T00:06:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-15T00:06:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:30.585 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-09-15T00:06:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.585 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:06:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T00:06:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T00:06:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T00:06:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T00:06:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T00:06:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T00:06:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:30.588 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:06:30.924 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-test 2024-09-15T00:06:31.522 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:30 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:06:31.661 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:06:31.661 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:31.661 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.1.1-324.gecab7be5.el9 ceph 49 M 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.1.1-324.gecab7be5.el9 ceph 163 k 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:Install 4 Packages 2024-09-15T00:06:31.662 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:31.663 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 49 M 2024-09-15T00:06:31.663 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 209 M 2024-09-15T00:06:31.663 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:06:31.814 INFO:teuthology.orchestra.run.smithi027.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.3 MB/s | 304 kB 00:00 2024-09-15T00:06:31.855 INFO:teuthology.orchestra.run.smithi027.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-15T00:06:31.914 INFO:teuthology.orchestra.run.smithi027.stdout:(3/4): libcephsqlite-19.1.1-324.gecab7be5.el9.x 719 kB/s | 163 kB 00:00 2024-09-15T00:06:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:06:33.231 INFO:teuthology.orchestra.run.smithi027.stdout:(4/4): ceph-test-19.1.1-324.gecab7be5.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-15T00:06:33.233 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:33.233 INFO:teuthology.orchestra.run.smithi027.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-15T00:06:33.250 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:06:33.262 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:06:33.284 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:06:33.284 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:06:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:06:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:06:33.676 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:06:33.677 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:06:34.222 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:06:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T00:06:34.343 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T00:06:34.374 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:06:35.616 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:06:35.616 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 1/4 2024-09-15T00:06:35.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2/4 2024-09-15T00:06:35.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T00:06:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:06:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 1/4 2024-09-15T00:06:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2/4 2024-09-15T00:06:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T00:06:35.972 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T00:06:35.972 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:35.972 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:06:35.972 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:35.973 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:35.973 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T00:06:35.973 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T00:06:35.973 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:35.973 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:06:36.029 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T00:06:36.030 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:36.030 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:06:36.030 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:36.030 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:36.030 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T00:06:36.030 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T00:06:36.030 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:36.030 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:06:36.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph 2024-09-15T00:06:36.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph 2024-09-15T00:06:36.836 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:40 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:06:36.871 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:42 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:06:36.976 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:06:36.978 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.1.1-324.gecab7be5.el9 ceph 6.5 k 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.1.1-324.gecab7be5.el9 ceph 2.4 M 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.1.1-324.gecab7be5.el9 ceph 1.5 M 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 249 k 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.1.1-324.gecab7be5.el9 ceph 4.7 M 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 17 M 2024-09-15T00:06:36.979 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T00:06:36.980 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T00:06:36.981 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout:Install 37 Packages 2024-09-15T00:06:36.982 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:36.983 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 30 M 2024-09-15T00:06:36.983 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 108 M 2024-09-15T00:06:36.983 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:06:37.014 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:06:37.017 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:06:37.017 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:37.017 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:06:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:19.1.1-324.gecab7be5.el9 ceph 6.5 k 2024-09-15T00:06:37.017 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-15T00:06:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:19.1.1-324.gecab7be5.el9 ceph 2.4 M 2024-09-15T00:06:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:19.1.1-324.gecab7be5.el9 ceph 1.5 M 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 249 k 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:19.1.1-324.gecab7be5.el9 ceph 4.7 M 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 17 M 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T00:06:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T00:06:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:37.020 INFO:teuthology.orchestra.run.smithi079.stdout:Install 37 Packages 2024-09-15T00:06:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:37.021 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 30 M 2024-09-15T00:06:37.021 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 108 M 2024-09-15T00:06:37.021 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:06:37.539 INFO:teuthology.orchestra.run.smithi079.stdout:(1/37): ceph-19.1.1-324.gecab7be5.el9.x86_64.rp 53 kB/s | 6.5 kB 00:00 2024-09-15T00:06:37.700 INFO:teuthology.orchestra.run.smithi092.stdout:(1/37): ceph-19.1.1-324.gecab7be5.el9.x86_64.rp 53 kB/s | 6.5 kB 00:00 2024-09-15T00:06:37.714 INFO:teuthology.orchestra.run.smithi079.stdout:(2/37): ceph-mgr-19.1.1-324.gecab7be5.el9.x86_6 5.1 MB/s | 1.5 MB 00:00 2024-09-15T00:06:37.789 INFO:teuthology.orchestra.run.smithi079.stdout:(3/37): ceph-mds-19.1.1-324.gecab7be5.el9.x86_6 6.4 MB/s | 2.4 MB 00:00 2024-09-15T00:06:37.906 INFO:teuthology.orchestra.run.smithi079.stdout:(4/37): ceph-mon-19.1.1-324.gecab7be5.el9.x86_6 13 MB/s | 4.7 MB 00:00 2024-09-15T00:06:37.931 INFO:teuthology.orchestra.run.smithi079.stdout:(5/37): ceph-mgr-modules-core-19.1.1-324.gecab7 1.7 MB/s | 249 kB 00:00 2024-09-15T00:06:38.074 INFO:teuthology.orchestra.run.smithi079.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-15T00:06:38.235 INFO:teuthology.orchestra.run.smithi092.stdout:(2/37): ceph-mgr-19.1.1-324.gecab7be5.el9.x86_6 2.3 MB/s | 1.5 MB 00:00 2024-09-15T00:06:38.317 INFO:teuthology.orchestra.run.smithi092.stdout:(3/37): ceph-mds-19.1.1-324.gecab7be5.el9.x86_6 3.2 MB/s | 2.4 MB 00:00 2024-09-15T00:06:38.324 INFO:teuthology.orchestra.run.smithi079.stdout:(7/37): ceph-osd-19.1.1-324.gecab7be5.el9.x86_6 28 MB/s | 17 MB 00:00 2024-09-15T00:06:38.349 INFO:teuthology.orchestra.run.smithi079.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 387 kB/s | 106 kB 00:00 2024-09-15T00:06:38.385 INFO:teuthology.orchestra.run.smithi092.stdout:(4/37): ceph-mgr-modules-core-19.1.1-324.gecab7 3.6 MB/s | 249 kB 00:00 2024-09-15T00:06:38.424 INFO:teuthology.orchestra.run.smithi079.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.6 MB/s | 125 kB 00:00 2024-09-15T00:06:38.483 INFO:teuthology.orchestra.run.smithi079.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 851 kB/s | 135 kB 00:00 2024-09-15T00:06:38.527 INFO:teuthology.orchestra.run.smithi092.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-15T00:06:38.533 INFO:teuthology.orchestra.run.smithi079.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-09-15T00:06:38.660 INFO:teuthology.orchestra.run.smithi092.stdout:(6/37): python3-cryptography-36.0.1-4.el9.x86_6 9.2 MB/s | 1.2 MB 00:00 2024-09-15T00:06:38.700 INFO:teuthology.orchestra.run.smithi079.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 868 kB/s | 188 kB 00:00 2024-09-15T00:06:38.710 INFO:teuthology.orchestra.run.smithi092.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-15T00:06:38.734 INFO:teuthology.orchestra.run.smithi079.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 859 kB/s | 172 kB 00:00 2024-09-15T00:06:38.753 INFO:teuthology.orchestra.run.smithi092.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-15T00:06:38.759 INFO:teuthology.orchestra.run.smithi079.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 593 kB/s | 35 kB 00:00 2024-09-15T00:06:38.793 INFO:teuthology.orchestra.run.smithi079.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.3 MB/s | 77 kB 00:00 2024-09-15T00:06:38.819 INFO:teuthology.orchestra.run.smithi092.stdout:(9/37): ceph-mon-19.1.1-324.gecab7be5.el9.x86_6 4.2 MB/s | 4.7 MB 00:01 2024-09-15T00:06:38.827 INFO:teuthology.orchestra.run.smithi079.stdout:(16/37): python3-cryptography-36.0.1-4.el9.x86_ 1.4 MB/s | 1.2 MB 00:00 2024-09-15T00:06:38.845 INFO:teuthology.orchestra.run.smithi092.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.3 MB/s | 125 kB 00:00 2024-09-15T00:06:38.852 INFO:teuthology.orchestra.run.smithi079.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 551 kB/s | 51 kB 00:00 2024-09-15T00:06:38.877 INFO:teuthology.orchestra.run.smithi079.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 497 kB/s | 42 kB 00:00 2024-09-15T00:06:38.919 INFO:teuthology.orchestra.run.smithi079.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-15T00:06:38.936 INFO:teuthology.orchestra.run.smithi079.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 11 MB/s | 172 kB 00:00 2024-09-15T00:06:38.962 INFO:teuthology.orchestra.run.smithi079.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 517 kB/s | 43 kB 00:00 2024-09-15T00:06:38.978 INFO:teuthology.orchestra.run.smithi092.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.3 MB/s | 215 kB 00:00 2024-09-15T00:06:38.978 INFO:teuthology.orchestra.run.smithi079.stdout:(22/37): lua-devel-5.4.4-4.el9.x86_64.rpm 147 kB/s | 22 kB 00:00 2024-09-15T00:06:38.995 INFO:teuthology.orchestra.run.smithi079.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 318 kB/s | 11 kB 00:00 2024-09-15T00:06:39.021 INFO:teuthology.orchestra.run.smithi079.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 1.3 MB/s | 23 kB 00:00 2024-09-15T00:06:39.038 INFO:teuthology.orchestra.run.smithi079.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 1.3 MB/s | 19 kB 00:00 2024-09-15T00:06:39.055 INFO:teuthology.orchestra.run.smithi079.stdout:(26/37): python3-jaraco-classes-3.2.1-5.el9.noa 233 kB/s | 18 kB 00:00 2024-09-15T00:06:39.072 INFO:teuthology.orchestra.run.smithi079.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 584 kB/s | 20 kB 00:00 2024-09-15T00:06:39.097 INFO:teuthology.orchestra.run.smithi079.stdout:(28/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-09-15T00:06:39.123 INFO:teuthology.orchestra.run.smithi079.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 688 kB/s | 46 kB 00:00 2024-09-15T00:06:39.148 INFO:teuthology.orchestra.run.smithi079.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-15T00:06:39.173 INFO:teuthology.orchestra.run.smithi079.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-15T00:06:39.198 INFO:teuthology.orchestra.run.smithi079.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-15T00:06:39.215 INFO:teuthology.orchestra.run.smithi079.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-09-15T00:06:39.241 INFO:teuthology.orchestra.run.smithi079.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-15T00:06:39.261 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-15T00:06:39.266 INFO:teuthology.orchestra.run.smithi079.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-09-15T00:06:39.291 INFO:teuthology.orchestra.run.smithi079.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-09-15T00:06:39.317 INFO:teuthology.orchestra.run.smithi079.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-15T00:06:39.322 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:39.322 INFO:teuthology.orchestra.run.smithi079.stdout:Total 13 MB/s | 30 MB 00:02 2024-09-15T00:06:39.343 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:06:39.387 INFO:teuthology.orchestra.run.smithi092.stdout:(12/37): ceph-osd-19.1.1-324.gecab7be5.el9.x86_ 15 MB/s | 17 MB 00:01 2024-09-15T00:06:39.438 INFO:teuthology.orchestra.run.smithi092.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 318 kB/s | 188 kB 00:00 2024-09-15T00:06:39.475 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:06:39.514 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:06:39.515 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:06:39.546 INFO:teuthology.orchestra.run.smithi092.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 713 kB/s | 77 kB 00:00 2024-09-15T00:06:39.588 INFO:teuthology.orchestra.run.smithi092.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 283 kB/s | 172 kB 00:00 2024-09-15T00:06:39.663 INFO:teuthology.orchestra.run.smithi092.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-09-15T00:06:39.688 INFO:teuthology.orchestra.run.smithi092.stdout:(17/37): python3-markupsafe-1.1.1-12.el9.x86_64 115 kB/s | 35 kB 00:00 2024-09-15T00:06:39.713 INFO:teuthology.orchestra.run.smithi092.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 332 kB/s | 42 kB 00:00 2024-09-15T00:06:39.756 INFO:teuthology.orchestra.run.smithi092.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-15T00:06:39.781 INFO:teuthology.orchestra.run.smithi092.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 650 kB/s | 43 kB 00:00 2024-09-15T00:06:39.806 INFO:teuthology.orchestra.run.smithi092.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.5 MB/s | 172 kB 00:00 2024-09-15T00:06:39.819 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:06:39.821 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:06:39.826 INFO:teuthology.orchestra.run.smithi092.stdout:(22/37): lua-devel-5.4.4-4.el9.x86_64.rpm 137 kB/s | 22 kB 00:00 2024-09-15T00:06:39.851 INFO:teuthology.orchestra.run.smithi092.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 238 kB/s | 11 kB 00:00 2024-09-15T00:06:39.885 INFO:teuthology.orchestra.run.smithi092.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 898 kB/s | 23 kB 00:00 2024-09-15T00:06:39.910 INFO:teuthology.orchestra.run.smithi092.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 836 kB/s | 19 kB 00:00 2024-09-15T00:06:39.944 INFO:teuthology.orchestra.run.smithi092.stdout:(26/37): python3-jaraco-classes-3.2.1-5.el9.noa 150 kB/s | 18 kB 00:00 2024-09-15T00:06:39.969 INFO:teuthology.orchestra.run.smithi092.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 335 kB/s | 20 kB 00:00 2024-09-15T00:06:39.994 INFO:teuthology.orchestra.run.smithi092.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-15T00:06:40.020 INFO:teuthology.orchestra.run.smithi092.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-15T00:06:40.045 INFO:teuthology.orchestra.run.smithi092.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-15T00:06:40.070 INFO:teuthology.orchestra.run.smithi092.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-15T00:06:40.087 INFO:teuthology.orchestra.run.smithi092.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-15T00:06:40.112 INFO:teuthology.orchestra.run.smithi092.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 855 kB/s | 36 kB 00:00 2024-09-15T00:06:40.138 INFO:teuthology.orchestra.run.smithi092.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-15T00:06:40.163 INFO:teuthology.orchestra.run.smithi092.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 805 kB/s | 20 kB 00:00 2024-09-15T00:06:40.188 INFO:teuthology.orchestra.run.smithi092.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-15T00:06:40.213 INFO:teuthology.orchestra.run.smithi092.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-15T00:06:40.218 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:40.218 INFO:teuthology.orchestra.run.smithi092.stdout:Total 9.4 MB/s | 30 MB 00:03 2024-09-15T00:06:40.367 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:06:40.406 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:06:40.406 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:06:40.720 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:06:40.720 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:06:41.192 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:06:41.261 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T00:06:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T00:06:41.322 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T00:06:41.394 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T00:06:41.438 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T00:06:41.471 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T00:06:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T00:06:41.633 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T00:06:41.667 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-15T00:06:41.668 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 1/4 2024-09-15T00:06:41.668 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2/4 2024-09-15T00:06:41.668 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T00:06:41.685 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T00:06:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T00:06:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T00:06:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T00:06:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T00:06:41.869 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T00:06:41.911 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T00:06:41.945 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T00:06:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T00:06:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:41.987 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:06:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T00:06:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T00:06:41.987 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:41.987 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:06:41.994 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T00:06:42.026 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T00:06:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:06:42.099 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T00:06:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T00:06:42.183 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T00:06:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T00:06:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T00:06:42.271 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph 2024-09-15T00:06:42.290 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T00:06:42.325 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T00:06:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T00:06:42.384 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T00:06:42.416 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T00:06:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T00:06:42.466 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T00:06:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T00:06:42.522 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T00:06:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T00:06:42.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T00:06:42.663 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T00:06:42.693 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T00:06:42.714 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T00:06:42.719 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T00:06:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T00:06:42.769 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T00:06:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T00:06:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T00:06:42.852 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T00:06:42.861 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:41 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:06:42.887 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T00:06:42.935 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T00:06:42.938 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T00:06:42.967 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T00:06:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T00:06:43.015 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:06:43.018 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:06:43.018 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:43.018 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:06:43.018 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.1.1-324.gecab7be5.el9 ceph 6.5 k 2024-09-15T00:06:43.018 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.1.1-324.gecab7be5.el9 ceph 2.4 M 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.1.1-324.gecab7be5.el9 ceph 1.5 M 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 249 k 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.1.1-324.gecab7be5.el9 ceph 4.7 M 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 17 M 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T00:06:43.019 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T00:06:43.020 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:43.021 INFO:teuthology.orchestra.run.smithi027.stdout:Install 37 Packages 2024-09-15T00:06:43.022 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:43.022 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 30 M 2024-09-15T00:06:43.022 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 108 M 2024-09-15T00:06:43.022 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:06:43.040 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T00:06:43.096 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T00:06:43.179 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T00:06:43.261 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 32/37 2024-09-15T00:06:43.266 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T00:06:43.291 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:06:43.315 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:06:43.316 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:43.316 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T00:06:43.316 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:06:43.316 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:06:43.316 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:43.328 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T00:06:43.367 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T00:06:43.417 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T00:06:43.479 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T00:06:43.503 INFO:teuthology.orchestra.run.smithi027.stdout:(1/37): ceph-19.1.1-324.gecab7be5.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-15T00:06:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T00:06:43.711 INFO:teuthology.orchestra.run.smithi027.stdout:(2/37): ceph-mds-19.1.1-324.gecab7be5.el9.x86_6 7.2 MB/s | 2.4 MB 00:00 2024-09-15T00:06:43.720 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T00:06:43.753 INFO:teuthology.orchestra.run.smithi027.stdout:(3/37): ceph-mgr-19.1.1-324.gecab7be5.el9.x86_6 4.1 MB/s | 1.5 MB 00:00 2024-09-15T00:06:43.806 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T00:06:43.862 INFO:teuthology.orchestra.run.smithi027.stdout:(4/37): ceph-mon-19.1.1-324.gecab7be5.el9.x86_6 13 MB/s | 4.7 MB 00:00 2024-09-15T00:06:43.887 INFO:teuthology.orchestra.run.smithi027.stdout:(5/37): ceph-mgr-modules-core-19.1.1-324.gecab7 1.8 MB/s | 249 kB 00:00 2024-09-15T00:06:43.896 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T00:06:43.936 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T00:06:44.037 INFO:teuthology.orchestra.run.smithi027.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-15T00:06:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T00:06:44.088 INFO:teuthology.orchestra.run.smithi027.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-15T00:06:44.224 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 32/37 2024-09-15T00:06:44.255 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:06:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:06:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T00:06:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:06:44.281 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:06:44.281 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:44.288 INFO:teuthology.orchestra.run.smithi027.stdout:(8/37): ceph-osd-19.1.1-324.gecab7be5.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-15T00:06:44.313 INFO:teuthology.orchestra.run.smithi027.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 600 kB/s | 135 kB 00:00 2024-09-15T00:06:44.347 INFO:teuthology.orchestra.run.smithi027.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.7 MB/s | 1.2 MB 00:00 2024-09-15T00:06:44.372 INFO:teuthology.orchestra.run.smithi027.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-15T00:06:44.431 INFO:teuthology.orchestra.run.smithi027.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 877 kB/s | 125 kB 00:00 2024-09-15T00:06:44.565 INFO:teuthology.orchestra.run.smithi027.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 260 kB/s | 35 kB 00:00 2024-09-15T00:06:44.598 INFO:teuthology.orchestra.run.smithi027.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 763 kB/s | 172 kB 00:00 2024-09-15T00:06:44.623 INFO:teuthology.orchestra.run.smithi027.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 683 kB/s | 188 kB 00:00 2024-09-15T00:06:44.657 INFO:teuthology.orchestra.run.smithi027.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-09-15T00:06:44.674 INFO:teuthology.orchestra.run.smithi027.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 708 kB/s | 77 kB 00:00 2024-09-15T00:06:44.690 INFO:teuthology.orchestra.run.smithi027.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 619 kB/s | 42 kB 00:00 2024-09-15T00:06:44.733 INFO:teuthology.orchestra.run.smithi027.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-15T00:06:44.758 INFO:teuthology.orchestra.run.smithi027.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 650 kB/s | 43 kB 00:00 2024-09-15T00:06:44.758 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:06:44.783 INFO:teuthology.orchestra.run.smithi027.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.5 MB/s | 172 kB 00:00 2024-09-15T00:06:44.785 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:06:44.785 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:44.785 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T00:06:44.785 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:06:44.786 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:06:44.786 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:44.809 INFO:teuthology.orchestra.run.smithi027.stdout:(22/37): lua-devel-5.4.4-4.el9.x86_64.rpm 147 kB/s | 22 kB 00:00 2024-09-15T00:06:44.834 INFO:teuthology.orchestra.run.smithi027.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 214 kB/s | 11 kB 00:00 2024-09-15T00:06:44.868 INFO:teuthology.orchestra.run.smithi027.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 917 kB/s | 23 kB 00:00 2024-09-15T00:06:44.893 INFO:teuthology.orchestra.run.smithi027.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 211 kB/s | 18 kB 00:00 2024-09-15T00:06:44.918 INFO:teuthology.orchestra.run.smithi027.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 405 kB/s | 19 kB 00:00 2024-09-15T00:06:44.943 INFO:teuthology.orchestra.run.smithi027.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 798 kB/s | 20 kB 00:00 2024-09-15T00:06:44.969 INFO:teuthology.orchestra.run.smithi027.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 927 kB/s | 46 kB 00:00 2024-09-15T00:06:44.994 INFO:teuthology.orchestra.run.smithi027.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-15T00:06:45.019 INFO:teuthology.orchestra.run.smithi027.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-15T00:06:45.044 INFO:teuthology.orchestra.run.smithi027.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-15T00:06:45.070 INFO:teuthology.orchestra.run.smithi027.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-15T00:06:45.095 INFO:teuthology.orchestra.run.smithi027.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-15T00:06:45.120 INFO:teuthology.orchestra.run.smithi027.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-15T00:06:45.145 INFO:teuthology.orchestra.run.smithi027.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-15T00:06:45.171 INFO:teuthology.orchestra.run.smithi027.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-15T00:06:45.196 INFO:teuthology.orchestra.run.smithi027.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-15T00:06:45.199 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:45.199 INFO:teuthology.orchestra.run.smithi027.stdout:Total 14 MB/s | 30 MB 00:02 2024-09-15T00:06:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:06:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:06:45.260 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:45.260 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T00:06:45.260 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:06:45.260 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:06:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:45.356 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:06:45.401 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:06:45.401 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:06:45.493 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:06:45.516 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:06:45.517 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:45.517 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T00:06:45.517 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:06:45.517 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:06:45.517 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:06:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:06:45.744 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:06:45.746 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:06:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:06:45.750 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:45.750 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T00:06:45.750 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:06:45.750 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:06:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:46.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:06:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:06:46.205 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:46.205 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T00:06:46.205 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:06:46.205 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:06:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:46.440 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:06:46.464 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:06:46.464 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:46.464 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T00:06:46.464 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:06:46.465 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:06:46.465 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:46.529 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:06:47.140 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:06:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T00:06:47.256 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T00:06:47.285 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T00:06:47.359 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T00:06:47.409 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T00:06:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T00:06:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T00:06:47.600 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:06:47.600 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 1/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 3/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 4/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 5/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 6/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T00:06:47.601 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T00:06:47.602 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T00:06:47.603 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T00:06:47.603 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T00:06:47.603 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T00:06:47.603 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T00:06:47.603 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T00:06:47.603 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T00:06:47.603 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T00:06:47.603 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T00:06:47.603 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T00:06:47.614 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T00:06:47.662 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T00:06:47.698 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T00:06:47.727 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T00:06:47.762 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T00:06:47.795 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T00:06:47.837 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T00:06:47.879 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T00:06:47.913 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T00:06:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T00:06:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T00:06:48.079 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 1/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 3/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 4/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 5/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 6/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T00:06:48.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T00:06:48.133 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T00:06:48.134 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T00:06:48.135 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T00:06:48.135 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T00:06:48.239 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T00:06:48.321 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T00:06:48.382 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T00:06:48.419 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T00:06:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T00:06:48.542 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T00:06:48.697 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T00:06:48.697 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T00:06:48.698 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T00:06:48.699 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T00:06:48.700 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T00:06:48.701 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:48.701 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:06:48.737 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T00:06:48.790 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T00:06:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T00:06:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T00:06:48.997 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T00:06:49.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-base 2024-09-15T00:06:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T00:06:49.116 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:49.117 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:06:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:06:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:49.117 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T00:06:49.118 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T00:06:49.119 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T00:06:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T00:06:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T00:06:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T00:06:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T00:06:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T00:06:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T00:06:49.120 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:06:49.121 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T00:06:49.292 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 32/37 2024-09-15T00:06:49.323 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:06:49.351 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-15T00:06:49.351 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:49.352 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T00:06:49.352 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:06:49.352 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T00:06:49.352 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:49.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-base 2024-09-15T00:06:49.651 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:55 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:06:49.741 INFO:teuthology.orchestra.run.smithi079.stdout:Package ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:06:49.783 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:06:49.783 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:06:49.784 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:06:49.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install cephadm 2024-09-15T00:06:50.083 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:54 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:06:50.172 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:06:50.213 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:06:50.214 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:06:50.215 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:06:50.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install cephadm 2024-09-15T00:06:50.405 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:56 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:06:50.537 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:06:50.537 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:50.537 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:06:50.537 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:50.538 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:06:50.538 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 765 k 2024-09-15T00:06:50.538 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:50.538 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:06:50.538 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:50.538 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-15T00:06:50.538 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:50.539 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 765 k 2024-09-15T00:06:50.539 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 771 k 2024-09-15T00:06:50.539 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:06:50.811 INFO:teuthology.orchestra.run.smithi079.stdout:cephadm-19.1.1-324.gecab7be5.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-15T00:06:50.812 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:50.812 INFO:teuthology.orchestra.run.smithi079.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-15T00:06:50.813 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:06:50.818 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:06:50.818 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:06:50.823 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:06:50.824 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:06:50.843 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:54 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:06:50.874 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:06:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:06:50.903 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-15T00:06:50.903 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:50.903 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T00:06:50.903 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:06:50.903 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T00:06:50.903 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:50.975 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:06:50.976 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:50.976 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:06:50.976 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:50.976 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:06:50.977 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 765 k 2024-09-15T00:06:50.977 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:50.977 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:06:50.977 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:50.977 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-15T00:06:50.977 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:50.978 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 765 k 2024-09-15T00:06:50.978 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 771 k 2024-09-15T00:06:50.978 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:06:51.254 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm-19.1.1-324.gecab7be5.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-15T00:06:51.255 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:51.256 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-15T00:06:51.256 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:06:51.260 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:06:51.260 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:06:51.266 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:06:51.266 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:06:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:06:51.358 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:06:51.383 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-15T00:06:51.383 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:51.383 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T00:06:51.383 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:06:51.383 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T00:06:51.383 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:51.617 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:06:51.642 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-15T00:06:51.642 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:51.642 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T00:06:51.642 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:06:51.642 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T00:06:51.642 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:51.709 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:06:51.739 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:51.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:52.041 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:06:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:06:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:52.041 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:06:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:52.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T00:06:52.467 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:52.467 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:52.467 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:06:52.468 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:06:52.468 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:52.468 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:06:52.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T00:06:52.851 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:58 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:06:52.986 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:06:52.986 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:52.986 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2024-09-15T00:06:52.986 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-324.gecab7be5.el9 ceph 145 k 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 145 k 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 435 k 2024-09-15T00:06:52.987 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:06:53.204 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-immutable-object-cache-19.1.1-324.gecab7be 670 kB/s | 145 kB 00:00 2024-09-15T00:06:53.204 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:53.205 INFO:teuthology.orchestra.run.smithi079.stdout:Total 665 kB/s | 145 kB 00:00 2024-09-15T00:06:53.205 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:06:53.212 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:06:53.212 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:06:53.270 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:06:53.270 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:06:53.272 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:57 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:06:53.406 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:06:53.407 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-15T00:06:53.407 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:53.407 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:06:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-324.gecab7be5.el9 ceph 145 k 2024-09-15T00:06:53.407 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:53.407 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:06:53.408 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:53.408 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-15T00:06:53.408 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:53.408 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 145 k 2024-09-15T00:06:53.408 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 435 k 2024-09-15T00:06:53.408 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:06:53.418 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-15T00:06:53.418 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 1/37 2024-09-15T00:06:53.418 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2/37 2024-09-15T00:06:53.418 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 3/37 2024-09-15T00:06:53.418 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 4/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 5/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 6/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T00:06:53.419 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T00:06:53.420 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T00:06:53.420 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T00:06:53.420 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T00:06:53.420 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T00:06:53.420 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T00:06:53.421 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T00:06:53.421 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T00:06:53.421 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T00:06:53.421 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:06:53.422 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T00:06:53.422 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T00:06:53.422 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T00:06:53.423 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T00:06:53.423 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T00:06:53.423 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T00:06:53.423 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T00:06:53.423 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T00:06:53.424 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T00:06:53.424 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T00:06:53.424 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T00:06:53.505 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:06:53.531 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:06:53.531 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:53.531 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T00:06:53.531 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:53.611 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-immutable-object-cache-19.1.1-324.gecab7be 714 kB/s | 145 kB 00:00 2024-09-15T00:06:53.612 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:53.612 INFO:teuthology.orchestra.run.smithi092.stdout:Total 708 kB/s | 145 kB 00:00 2024-09-15T00:06:53.612 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:06:53.620 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:06:53.620 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:06:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:06:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:06:53.834 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:06:53.921 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:06:53.946 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:06:53.946 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:53.946 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T00:06:53.946 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T00:06:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:54.364 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:06:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:54.364 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T00:06:54.365 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T00:06:54.366 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T00:06:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T00:06:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T00:06:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T00:06:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T00:06:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T00:06:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T00:06:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T00:06:54.367 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:54.367 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:06:54.407 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:06:54.407 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:54.408 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:06:54.408 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:54.408 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:54.408 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:06:54.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr 2024-09-15T00:06:54.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-base 2024-09-15T00:06:54.861 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:06:54.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:54.861 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:06:54.861 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:06:54.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:54.862 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:06:55.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr 2024-09-15T00:06:55.235 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:01 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:06:55.328 INFO:teuthology.orchestra.run.smithi079.stdout:Package ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:06:55.347 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:54 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:06:55.371 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:06:55.372 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:06:55.372 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:06:55.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T00:06:55.442 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:06:55.485 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:06:55.486 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:06:55.486 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:06:55.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install cephadm 2024-09-15T00:06:55.659 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:59 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:06:55.750 INFO:teuthology.orchestra.run.smithi092.stdout:Package ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:06:55.792 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:06:55.793 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:06:55.793 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:06:55.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T00:06:56.007 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:02 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:06:56.129 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:55 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:06:56.141 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:06:56.142 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 3.6 M 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 32 k 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 16 k 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T00:06:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T00:06:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T00:06:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T00:06:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T00:06:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:56.144 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:06:56.144 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:06:56.144 INFO:teuthology.orchestra.run.smithi079.stdout:Install 12 Packages 2024-09-15T00:06:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:06:56.145 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 8.7 M 2024-09-15T00:06:56.145 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 104 M 2024-09-15T00:06:56.145 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:06:56.267 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:06:56.267 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 765 k 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 765 k 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 771 k 2024-09-15T00:06:56.268 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:06:56.438 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:06:56.535 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm-19.1.1-324.gecab7be5.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-15T00:06:56.535 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:56.536 INFO:teuthology.orchestra.run.smithi027.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-15T00:06:56.536 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:06:56.541 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:06:56.542 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:06:56.547 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:06:56.548 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:06:56.569 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:06:56.570 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 3.6 M 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 32 k 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 16 k 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T00:06:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T00:06:56.572 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T00:06:56.572 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T00:06:56.572 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T00:06:56.572 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T00:06:56.572 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:56.572 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:06:56.572 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:06:56.572 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-09-15T00:06:56.572 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:06:56.573 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 8.7 M 2024-09-15T00:06:56.573 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 104 M 2024-09-15T00:06:56.573 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:06:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:06:56.981 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): ceph-grafana-dashboards-19.1.1-324.geca 214 kB/s | 32 kB 00:00 2024-09-15T00:06:56.985 INFO:teuthology.orchestra.run.smithi079.stdout:(1/12): ceph-grafana-dashboards-19.1.1-324.geca 194 kB/s | 32 kB 00:00 2024-09-15T00:06:56.998 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): ceph-prometheus-alerts-19.1.1-324.gecab 99 kB/s | 16 kB 00:00 2024-09-15T00:06:57.010 INFO:teuthology.orchestra.run.smithi079.stdout:(2/12): ceph-prometheus-alerts-19.1.1-324.gecab 87 kB/s | 16 kB 00:00 2024-09-15T00:06:57.107 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:57.183 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:57.210 INFO:teuthology.orchestra.run.smithi079.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-09-15T00:06:57.273 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): ceph-mgr-dashboard-19.1.1-324.gecab7be5 8.1 MB/s | 3.6 MB 00:00 2024-09-15T00:06:57.319 INFO:teuthology.orchestra.run.smithi079.stdout:(4/12): ceph-mgr-dashboard-19.1.1-324.gecab7be5 7.1 MB/s | 3.6 MB 00:00 2024-09-15T00:06:57.353 INFO:teuthology.orchestra.run.smithi079.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-09-15T00:06:57.403 INFO:teuthology.orchestra.run.smithi079.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 6.4 MB/s | 548 kB 00:00 2024-09-15T00:06:57.429 INFO:teuthology.orchestra.run.smithi079.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 256 kB/s | 19 kB 00:00 2024-09-15T00:06:57.462 INFO:teuthology.orchestra.run.smithi079.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-15T00:06:57.488 INFO:teuthology.orchestra.run.smithi079.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-15T00:06:57.504 INFO:teuthology.orchestra.run.smithi079.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 11 MB/s | 188 kB 00:00 2024-09-15T00:06:57.515 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 521 kB/s | 269 kB 00:00 2024-09-15T00:06:57.555 INFO:teuthology.orchestra.run.smithi079.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 13 MB/s | 2.0 MB 00:00 2024-09-15T00:06:57.582 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 9.0 MB/s | 548 kB 00:00 2024-09-15T00:06:57.599 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-15T00:06:57.658 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 35 MB/s | 2.0 MB 00:00 2024-09-15T00:06:57.678 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:57.683 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-15T00:06:57.708 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-15T00:06:57.733 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-15T00:06:57.767 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2024-09-15T00:06:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:06:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:57.947 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:06:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:06:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:57.947 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:06:57.963 INFO:teuthology.orchestra.run.smithi079.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.1 MB/s | 863 kB 00:00 2024-09-15T00:06:57.967 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:57.968 INFO:teuthology.orchestra.run.smithi079.stdout:Total 4.8 MB/s | 8.7 MB 00:01 2024-09-15T00:06:58.018 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:06:58.036 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:06:58.036 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:06:58.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T00:06:58.199 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:06:58.200 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:06:58.226 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 907 kB/s | 863 kB 00:00 2024-09-15T00:06:58.230 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:58.230 INFO:teuthology.orchestra.run.smithi092.stdout:Total 5.2 MB/s | 8.7 MB 00:01 2024-09-15T00:06:58.279 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:06:58.297 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:06:58.297 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:06:58.458 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:06:58.459 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:06:58.717 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:57 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:06:58.733 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:06:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-15T00:06:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-15T00:06:58.857 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:06:58.858 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:58.858 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-09-15T00:06:58.858 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:58.858 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:06:58.858 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-324.gecab7be5.el9 ceph 145 k 2024-09-15T00:06:58.858 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:58.858 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:06:58.859 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:06:58.859 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-15T00:06:58.859 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:58.859 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 145 k 2024-09-15T00:06:58.859 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 435 k 2024-09-15T00:06:58.859 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:06:58.896 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-15T00:06:58.927 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-15T00:06:58.998 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:06:59.041 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-15T00:06:59.074 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-immutable-object-cache-19.1.1-324.gecab7be 678 kB/s | 145 kB 00:00 2024-09-15T00:06:59.074 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:06:59.075 INFO:teuthology.orchestra.run.smithi027.stdout:Total 673 kB/s | 145 kB 00:00 2024-09-15T00:06:59.075 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:06:59.083 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:06:59.083 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:06:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-15T00:06:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-15T00:06:59.143 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:06:59.144 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:06:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-15T00:06:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-15T00:06:59.214 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-15T00:06:59.264 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-15T00:06:59.297 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:06:59.315 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-15T00:06:59.332 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-15T00:06:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:06:59.358 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:06:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 10/12 2024-09-15T00:06:59.384 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:06:59.385 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:06:59.385 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T00:06:59.385 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:06:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-15T00:06:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-15T00:06:59.593 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-15T00:06:59.621 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:06:59.653 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 10/12 2024-09-15T00:07:00.280 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-15T00:07:00.280 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:00.280 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:00.281 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:00.281 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:00.281 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 11/12 2024-09-15T00:07:00.468 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:07:00.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr 2024-09-15T00:07:00.780 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 11/12 2024-09-15T00:07:00.820 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:07:01.096 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:01.191 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:01.235 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:01.236 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:07:01.237 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:01.299 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T00:07:01.886 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:01.957 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:07:01.957 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 1/12 2024-09-15T00:07:01.957 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 2/12 2024-09-15T00:07:01.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 3/12 2024-09-15T00:07:01.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-15T00:07:01.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-15T00:07:01.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-15T00:07:01.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-15T00:07:01.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-15T00:07:01.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:07:01.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-15T00:07:01.958 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-15T00:07:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:07:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 1/12 2024-09-15T00:07:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 2/12 2024-09-15T00:07:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 3/12 2024-09-15T00:07:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-15T00:07:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-15T00:07:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-15T00:07:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-15T00:07:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-15T00:07:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:07:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-15T00:07:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-15T00:07:02.025 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:02.027 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:02.027 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:07:02.027 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:02.027 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:02.027 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 3.6 M 2024-09-15T00:07:02.027 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-15T00:07:02.027 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T00:07:02.027 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 32 k 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 16 k 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T00:07:02.028 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:02.029 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:02.029 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:02.029 INFO:teuthology.orchestra.run.smithi027.stdout:Install 12 Packages 2024-09-15T00:07:02.029 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:02.029 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 8.7 M 2024-09-15T00:07:02.030 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 104 M 2024-09-15T00:07:02.030 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T00:07:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T00:07:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T00:07:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T00:07:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T00:07:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T00:07:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:02.282 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:02.325 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-15T00:07:02.325 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:02.325 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:02.325 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T00:07:02.325 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:02.325 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:02.325 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:02.325 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T00:07:02.325 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T00:07:02.326 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T00:07:02.326 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T00:07:02.326 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T00:07:02.326 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T00:07:02.326 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T00:07:02.326 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T00:07:02.326 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:02.326 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:02.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T00:07:02.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T00:07:02.853 INFO:teuthology.orchestra.run.smithi027.stdout:(1/12): ceph-grafana-dashboards-19.1.1-324.geca 223 kB/s | 32 kB 00:00 2024-09-15T00:07:02.878 INFO:teuthology.orchestra.run.smithi027.stdout:(2/12): ceph-prometheus-alerts-19.1.1-324.gecab 98 kB/s | 16 kB 00:00 2024-09-15T00:07:03.054 INFO:teuthology.orchestra.run.smithi027.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 269 kB 00:00 2024-09-15T00:07:03.104 INFO:teuthology.orchestra.run.smithi027.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-09-15T00:07:03.114 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:09 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:03.125 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:07 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:03.179 INFO:teuthology.orchestra.run.smithi027.stdout:(5/12): ceph-mgr-dashboard-19.1.1-324.gecab7be5 7.6 MB/s | 3.6 MB 00:00 2024-09-15T00:07:03.221 INFO:teuthology.orchestra.run.smithi027.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.6 MB/s | 548 kB 00:00 2024-09-15T00:07:03.246 INFO:teuthology.orchestra.run.smithi027.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 301 kB/s | 19 kB 00:00 2024-09-15T00:07:03.257 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:03.258 INFO:teuthology.orchestra.run.smithi079.stdout:====================================================================================== 2024-09-15T00:07:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:07:03.259 INFO:teuthology.orchestra.run.smithi079.stdout:====================================================================================== 2024-09-15T00:07:03.259 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 7.4 M 2024-09-15T00:07:03.259 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-15T00:07:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T00:07:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T00:07:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T00:07:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T00:07:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T00:07:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T00:07:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T00:07:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T00:07:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T00:07:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T00:07:03.261 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T00:07:03.261 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:03.261 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:03.261 INFO:teuthology.orchestra.run.smithi079.stdout:====================================================================================== 2024-09-15T00:07:03.261 INFO:teuthology.orchestra.run.smithi079.stdout:Install 12 Packages 2024-09-15T00:07:03.261 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:03.263 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 38 M 2024-09-15T00:07:03.263 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 212 M 2024-09-15T00:07:03.263 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:03.263 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-09-15T00:07:03.263 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:07:03.263 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-09-15T00:07:03.263 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:03.264 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 7.4 M 2024-09-15T00:07:03.264 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-15T00:07:03.264 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T00:07:03.264 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T00:07:03.264 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T00:07:03.264 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T00:07:03.264 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T00:07:03.265 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T00:07:03.265 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T00:07:03.265 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T00:07:03.265 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T00:07:03.265 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T00:07:03.268 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T00:07:03.268 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:03.268 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:03.268 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-09-15T00:07:03.268 INFO:teuthology.orchestra.run.smithi092.stdout:Install 12 Packages 2024-09-15T00:07:03.268 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:03.269 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 38 M 2024-09-15T00:07:03.269 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 212 M 2024-09-15T00:07:03.269 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:03.280 INFO:teuthology.orchestra.run.smithi027.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-15T00:07:03.305 INFO:teuthology.orchestra.run.smithi027.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-15T00:07:03.339 INFO:teuthology.orchestra.run.smithi027.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-15T00:07:03.372 INFO:teuthology.orchestra.run.smithi027.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 14 MB/s | 2.0 MB 00:00 2024-09-15T00:07:03.458 INFO:teuthology.orchestra.run.smithi092.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-15T00:07:03.489 INFO:teuthology.orchestra.run.smithi027.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.9 MB/s | 863 kB 00:00 2024-09-15T00:07:03.493 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:03.493 INFO:teuthology.orchestra.run.smithi027.stdout:Total 5.9 MB/s | 8.7 MB 00:01 2024-09-15T00:07:03.525 INFO:teuthology.orchestra.run.smithi092.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-15T00:07:03.545 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:03.563 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:03.563 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:03.641 INFO:teuthology.orchestra.run.smithi079.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-15T00:07:03.675 INFO:teuthology.orchestra.run.smithi079.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.7 MB/s | 801 kB 00:00 2024-09-15T00:07:03.734 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:03.734 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:03.767 INFO:teuthology.orchestra.run.smithi079.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-15T00:07:03.817 INFO:teuthology.orchestra.run.smithi079.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-09-15T00:07:03.876 INFO:teuthology.orchestra.run.smithi079.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-15T00:07:03.908 INFO:teuthology.orchestra.run.smithi092.stdout:(3/12): ceph-mgr-diskprediction-local-19.1.1-32 13 MB/s | 7.4 MB 00:00 2024-09-15T00:07:03.968 INFO:teuthology.orchestra.run.smithi079.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-09-15T00:07:04.018 INFO:teuthology.orchestra.run.smithi079.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-15T00:07:04.110 INFO:teuthology.orchestra.run.smithi079.stdout:(8/12): ceph-mgr-diskprediction-local-19.1.1-32 12 MB/s | 7.4 MB 00:00 2024-09-15T00:07:04.117 INFO:teuthology.orchestra.run.smithi092.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 50 kB/s | 33 kB 00:00 2024-09-15T00:07:04.211 INFO:teuthology.orchestra.run.smithi079.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-09-15T00:07:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:04.294 INFO:teuthology.orchestra.run.smithi079.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 12 MB/s | 4.9 MB 00:00 2024-09-15T00:07:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-15T00:07:04.384 INFO:teuthology.orchestra.run.smithi092.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 38 kB/s | 18 kB 00:00 2024-09-15T00:07:04.393 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-15T00:07:04.403 INFO:teuthology.orchestra.run.smithi079.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 857 kB/s | 250 kB 00:00 2024-09-15T00:07:04.440 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-15T00:07:04.477 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-15T00:07:04.606 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-15T00:07:04.645 INFO:teuthology.orchestra.run.smithi079.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-15T00:07:04.646 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:04.646 INFO:teuthology.orchestra.run.smithi079.stdout:Total 28 MB/s | 38 MB 00:01 2024-09-15T00:07:04.651 INFO:teuthology.orchestra.run.smithi092.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.7 MB/s | 3.0 MB 00:01 2024-09-15T00:07:04.693 INFO:teuthology.orchestra.run.smithi092.stdout:(7/12): openblas-0.3.26-2.el9.x86_64.rpm 67 kB/s | 38 kB 00:00 2024-09-15T00:07:04.736 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-15T00:07:04.835 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-15T00:07:04.875 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-15T00:07:04.902 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:07:04.934 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 10/12 2024-09-15T00:07:04.994 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:05.019 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:05.019 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:05.446 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:05.446 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:05.485 INFO:teuthology.orchestra.run.smithi092.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 296 kB/s | 245 kB 00:00 2024-09-15T00:07:06.052 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 11/12 2024-09-15T00:07:06.085 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:07:06.209 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:06.284 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T00:07:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:07:06.360 INFO:teuthology.orchestra.run.smithi092.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 286 kB/s | 250 kB 00:00 2024-09-15T00:07:06.433 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:07:06.478 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T00:07:06.662 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T00:07:06.702 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T00:07:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T00:07:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-15T00:07:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 1/12 2024-09-15T00:07:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 2/12 2024-09-15T00:07:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 3/12 2024-09-15T00:07:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-15T00:07:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-15T00:07:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-15T00:07:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-15T00:07:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-15T00:07:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-15T00:07:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-15T00:07:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-15T00:07:07.449 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T00:07:07.460 INFO:teuthology.orchestra.run.smithi092.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 1.6 MB/s | 4.9 MB 00:03 2024-09-15T00:07:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T00:07:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-15T00:07:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:07.764 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T00:07:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T00:07:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T00:07:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T00:07:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T00:07:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T00:07:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T00:07:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T00:07:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T00:07:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:07.767 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:07.818 INFO:teuthology.orchestra.run.smithi092.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.6 MB/s | 5.1 MB 00:03 2024-09-15T00:07:07.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T00:07:08.578 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:07 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:08.718 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:08.719 INFO:teuthology.orchestra.run.smithi027.stdout:====================================================================================== 2024-09-15T00:07:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:07:08.720 INFO:teuthology.orchestra.run.smithi027.stdout:====================================================================================== 2024-09-15T00:07:08.720 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 7.4 M 2024-09-15T00:07:08.720 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-15T00:07:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T00:07:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T00:07:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T00:07:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T00:07:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T00:07:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T00:07:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T00:07:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T00:07:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T00:07:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T00:07:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T00:07:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:08.722 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:08.722 INFO:teuthology.orchestra.run.smithi027.stdout:====================================================================================== 2024-09-15T00:07:08.722 INFO:teuthology.orchestra.run.smithi027.stdout:Install 12 Packages 2024-09-15T00:07:08.722 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:08.723 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 38 M 2024-09-15T00:07:08.723 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 212 M 2024-09-15T00:07:08.724 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:08.829 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:07:08.960 INFO:teuthology.orchestra.run.smithi092.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.3 MB/s | 16 MB 00:02 2024-09-15T00:07:08.961 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:08.961 INFO:teuthology.orchestra.run.smithi092.stdout:Total 6.7 MB/s | 38 MB 00:05 2024-09-15T00:07:09.224 INFO:teuthology.orchestra.run.smithi027.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-15T00:07:09.281 INFO:teuthology.orchestra.run.smithi027.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 801 kB 00:00 2024-09-15T00:07:09.315 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:09.340 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:09.340 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:09.573 INFO:teuthology.orchestra.run.smithi027.stdout:(3/12): ceph-mgr-diskprediction-local-19.1.1-32 15 MB/s | 7.4 MB 00:00 2024-09-15T00:07:09.763 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:09.764 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:09.857 INFO:teuthology.orchestra.run.smithi027.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-09-15T00:07:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T00:07:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:07:10.065 INFO:teuthology.orchestra.run.smithi027.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 38 kB/s | 18 kB 00:00 2024-09-15T00:07:10.415 INFO:teuthology.orchestra.run.smithi027.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-15T00:07:10.516 INFO:teuthology.orchestra.run.smithi027.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2024-09-15T00:07:10.530 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:10.614 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T00:07:10.665 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:07:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:07:10.800 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T00:07:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T00:07:11.024 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T00:07:11.101 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T00:07:11.191 INFO:teuthology.orchestra.run.smithi027.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.4 MB/s | 4.9 MB 00:01 2024-09-15T00:07:11.216 INFO:teuthology.orchestra.run.smithi027.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 306 kB/s | 245 kB 00:00 2024-09-15T00:07:11.692 INFO:teuthology.orchestra.run.smithi027.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-09-15T00:07:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T00:07:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T00:07:12.108 INFO:teuthology.orchestra.run.smithi027.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 273 kB/s | 250 kB 00:00 2024-09-15T00:07:12.469 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 1/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:07:12.470 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T00:07:12.774 INFO:teuthology.orchestra.run.smithi027.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-15T00:07:12.776 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:12.776 INFO:teuthology.orchestra.run.smithi027.stdout:Total 9.4 MB/s | 38 MB 00:04 2024-09-15T00:07:12.856 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T00:07:12.856 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:12.856 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:12.857 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:12.857 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T00:07:12.857 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T00:07:12.857 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T00:07:12.857 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T00:07:12.857 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T00:07:12.858 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T00:07:12.858 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T00:07:12.858 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T00:07:12.858 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T00:07:12.858 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T00:07:12.858 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T00:07:12.858 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:12.859 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:07:13.149 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:13.176 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:13.176 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:13.275 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-rook 2024-09-15T00:07:13.626 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:13.626 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:13.856 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:19 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:14.000 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:14.001 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:07:14.001 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:14.001 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 49 k 2024-09-15T00:07:14.001 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-15T00:07:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T00:07:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T00:07:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:14.002 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:14.003 INFO:teuthology.orchestra.run.smithi079.stdout:Install 13 Packages 2024-09-15T00:07:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:14.003 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 2.2 M 2024-09-15T00:07:14.003 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 26 M 2024-09-15T00:07:14.003 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:14.155 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T00:07:14.197 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:07:14.416 INFO:teuthology.orchestra.run.smithi079.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 200 kB/s | 19 kB 00:00 2024-09-15T00:07:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:14.441 INFO:teuthology.orchestra.run.smithi079.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 211 kB/s | 26 kB 00:00 2024-09-15T00:07:14.500 INFO:teuthology.orchestra.run.smithi079.stdout:(3/13): ceph-mgr-rook-19.1.1-324.gecab7be5.el9. 274 kB/s | 49 kB 00:00 2024-09-15T00:07:14.517 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T00:07:14.525 INFO:teuthology.orchestra.run.smithi079.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-15T00:07:14.561 INFO:teuthology.orchestra.run.smithi079.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-15T00:07:14.565 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:07:14.593 INFO:teuthology.orchestra.run.smithi079.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 800 kB/s | 54 kB 00:00 2024-09-15T00:07:14.626 INFO:teuthology.orchestra.run.smithi079.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-15T00:07:14.652 INFO:teuthology.orchestra.run.smithi079.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 246 kB/s | 14 kB 00:00 2024-09-15T00:07:14.674 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:07:14.677 INFO:teuthology.orchestra.run.smithi079.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-15T00:07:14.703 INFO:teuthology.orchestra.run.smithi079.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-15T00:07:14.725 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T00:07:14.728 INFO:teuthology.orchestra.run.smithi079.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-15T00:07:14.753 INFO:teuthology.orchestra.run.smithi079.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-15T00:07:14.795 INFO:teuthology.orchestra.run.smithi079.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-09-15T00:07:14.797 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:14.797 INFO:teuthology.orchestra.run.smithi079.stdout:Total 2.8 MB/s | 2.2 MB 00:00 2024-09-15T00:07:14.862 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:14.877 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:14.877 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T00:07:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T00:07:14.964 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:14.965 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T00:07:15.478 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:15.651 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T00:07:15.705 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T00:07:15.735 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T00:07:15.760 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T00:07:15.798 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T00:07:15.839 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T00:07:15.868 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T00:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T00:07:16.043 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T00:07:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T00:07:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:07:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 1/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:07:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T00:07:16.808 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T00:07:16.856 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T00:07:16.898 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T00:07:16.946 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T00:07:16.985 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T00:07:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T00:07:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T00:07:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T00:07:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T00:07:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T00:07:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T00:07:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:17.109 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:17.215 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:07:17.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-rook 2024-09-15T00:07:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:07:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 1/13 2024-09-15T00:07:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T00:07:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T00:07:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T00:07:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T00:07:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T00:07:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T00:07:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T00:07:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T00:07:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T00:07:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T00:07:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T00:07:18.044 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:22 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:18.119 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T00:07:18.119 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T00:07:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T00:07:18.121 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T00:07:18.121 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T00:07:18.121 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:18.121 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:18.175 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:18.176 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 49 k 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T00:07:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T00:07:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T00:07:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T00:07:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T00:07:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:18.178 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:18.178 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:18.178 INFO:teuthology.orchestra.run.smithi092.stdout:Install 13 Packages 2024-09-15T00:07:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:18.179 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 2.2 M 2024-09-15T00:07:18.179 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 26 M 2024-09-15T00:07:18.179 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:18.310 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T00:07:18.339 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:07:18.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T00:07:18.498 INFO:teuthology.orchestra.run.smithi092.stdout:(1/13): ceph-mgr-rook-19.1.1-324.gecab7be5.el9. 306 kB/s | 49 kB 00:00 2024-09-15T00:07:18.623 INFO:teuthology.orchestra.run.smithi092.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 67 kB/s | 19 kB 00:00 2024-09-15T00:07:18.648 INFO:teuthology.orchestra.run.smithi092.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 82 kB/s | 26 kB 00:00 2024-09-15T00:07:18.977 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:24 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:18.999 INFO:teuthology.orchestra.run.smithi092.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 423 kB/s | 159 kB 00:00 2024-09-15T00:07:19.024 INFO:teuthology.orchestra.run.smithi092.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 743 kB/s | 279 kB 00:00 2024-09-15T00:07:19.049 INFO:teuthology.orchestra.run.smithi092.stdout:(6/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 403 kB/s | 222 kB 00:00 2024-09-15T00:07:19.091 INFO:teuthology.orchestra.run.smithi092.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-15T00:07:19.111 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:19.112 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:19.112 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:07:19.112 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:19.112 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 146 k 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout:Install 6 Packages 2024-09-15T00:07:19.113 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:19.114 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 7.0 M 2024-09-15T00:07:19.114 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 33 M 2024-09-15T00:07:19.114 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:19.116 INFO:teuthology.orchestra.run.smithi092.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 457 kB/s | 54 kB 00:00 2024-09-15T00:07:19.141 INFO:teuthology.orchestra.run.smithi092.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 153 kB/s | 14 kB 00:00 2024-09-15T00:07:19.175 INFO:teuthology.orchestra.run.smithi092.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-15T00:07:19.209 INFO:teuthology.orchestra.run.smithi092.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-15T00:07:19.234 INFO:teuthology.orchestra.run.smithi092.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-15T00:07:19.268 INFO:teuthology.orchestra.run.smithi092.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-15T00:07:19.270 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:19.270 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-09-15T00:07:19.332 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:19.347 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:19.347 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:19.437 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:19.438 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:19.484 INFO:teuthology.orchestra.run.smithi079.stdout:(1/6): ceph-mgr-cephadm-19.1.1-324.gecab7be5.el 676 kB/s | 146 kB 00:00 2024-09-15T00:07:19.534 INFO:teuthology.orchestra.run.smithi079.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-15T00:07:19.560 INFO:teuthology.orchestra.run.smithi079.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-15T00:07:19.585 INFO:teuthology.orchestra.run.smithi079.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-15T00:07:19.902 INFO:teuthology.orchestra.run.smithi079.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 391 kB/s | 248 kB 00:00 2024-09-15T00:07:19.942 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:20.126 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T00:07:20.186 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T00:07:20.241 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T00:07:20.278 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T00:07:20.319 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T00:07:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T00:07:20.523 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T00:07:20.589 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T00:07:20.752 INFO:teuthology.orchestra.run.smithi079.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.0 MB/s | 6.0 MB 00:01 2024-09-15T00:07:20.754 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:20.754 INFO:teuthology.orchestra.run.smithi079.stdout:Total 4.3 MB/s | 7.0 MB 00:01 2024-09-15T00:07:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-15T00:07:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 1/12 2024-09-15T00:07:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T00:07:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T00:07:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T00:07:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T00:07:20.824 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T00:07:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T00:07:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T00:07:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T00:07:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T00:07:20.825 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T00:07:20.826 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:20.837 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:20.837 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:20.936 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:20.938 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:21.239 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:21.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T00:07:21.299 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T00:07:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T00:07:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T00:07:21.300 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T00:07:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T00:07:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T00:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T00:07:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T00:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T00:07:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:07:21.670 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-rook 2024-09-15T00:07:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T00:07:22.074 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T00:07:22.139 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T00:07:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:07:22.207 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:07:22.207 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 1/13 2024-09-15T00:07:22.207 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T00:07:22.208 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T00:07:22.259 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:21 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:22.401 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:22.402 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 49 k 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T00:07:22.403 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout:Install 13 Packages 2024-09-15T00:07:22.404 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:22.405 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 2.2 M 2024-09-15T00:07:22.405 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 26 M 2024-09-15T00:07:22.405 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T00:07:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T00:07:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T00:07:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T00:07:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T00:07:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T00:07:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T00:07:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T00:07:22.529 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:22.529 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:22.590 INFO:teuthology.orchestra.run.smithi027.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 220 kB/s | 19 kB 00:00 2024-09-15T00:07:22.616 INFO:teuthology.orchestra.run.smithi027.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 228 kB/s | 26 kB 00:00 2024-09-15T00:07:22.674 INFO:teuthology.orchestra.run.smithi027.stdout:(3/13): ceph-mgr-rook-19.1.1-324.gecab7be5.el9. 288 kB/s | 49 kB 00:00 2024-09-15T00:07:22.758 INFO:teuthology.orchestra.run.smithi027.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-15T00:07:22.783 INFO:teuthology.orchestra.run.smithi027.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.1 MB/s | 222 kB 00:00 2024-09-15T00:07:22.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T00:07:22.817 INFO:teuthology.orchestra.run.smithi027.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 912 kB/s | 54 kB 00:00 2024-09-15T00:07:22.842 INFO:teuthology.orchestra.run.smithi027.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 550 kB/s | 32 kB 00:00 2024-09-15T00:07:22.859 INFO:teuthology.orchestra.run.smithi027.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 353 kB/s | 14 kB 00:00 2024-09-15T00:07:22.884 INFO:teuthology.orchestra.run.smithi027.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 5.4 MB/s | 220 kB 00:00 2024-09-15T00:07:22.910 INFO:teuthology.orchestra.run.smithi027.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-15T00:07:22.926 INFO:teuthology.orchestra.run.smithi027.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-15T00:07:22.968 INFO:teuthology.orchestra.run.smithi027.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 9.4 MB/s | 1.0 MB 00:00 2024-09-15T00:07:23.002 INFO:teuthology.orchestra.run.smithi027.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 977 kB/s | 90 kB 00:00 2024-09-15T00:07:23.004 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:23.004 INFO:teuthology.orchestra.run.smithi027.stdout:Total 3.7 MB/s | 2.2 MB 00:00 2024-09-15T00:07:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:07:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/6 2024-09-15T00:07:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T00:07:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T00:07:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T00:07:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T00:07:23.069 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:23.085 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:23.085 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:23.182 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:23.182 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T00:07:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:23.294 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T00:07:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T00:07:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T00:07:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T00:07:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T00:07:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:23.295 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:23.352 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:27 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:23.487 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:23.488 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:23.488 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:07:23.488 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:23.488 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:23.488 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 146 k 2024-09-15T00:07:23.488 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-15T00:07:23.488 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T00:07:23.488 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T00:07:23.488 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T00:07:23.489 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T00:07:23.489 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T00:07:23.489 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:23.489 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:23.489 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:23.489 INFO:teuthology.orchestra.run.smithi092.stdout:Install 6 Packages 2024-09-15T00:07:23.489 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:23.490 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 7.0 M 2024-09-15T00:07:23.490 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 33 M 2024-09-15T00:07:23.490 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:23.576 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-fuse 2024-09-15T00:07:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T00:07:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T00:07:23.952 INFO:teuthology.orchestra.run.smithi092.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-15T00:07:23.977 INFO:teuthology.orchestra.run.smithi092.stdout:(2/6): ceph-mgr-cephadm-19.1.1-324.gecab7be5.el 688 kB/s | 146 kB 00:00 2024-09-15T00:07:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T00:07:24.027 INFO:teuthology.orchestra.run.smithi092.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-15T00:07:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T00:07:24.053 INFO:teuthology.orchestra.run.smithi092.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 766 kB/s | 58 kB 00:00 2024-09-15T00:07:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T00:07:24.078 INFO:teuthology.orchestra.run.smithi092.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-15T00:07:24.141 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:30 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:24.178 INFO:teuthology.orchestra.run.smithi092.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-15T00:07:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T00:07:24.181 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:24.181 INFO:teuthology.orchestra.run.smithi092.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-09-15T00:07:24.253 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:24.264 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:24.264 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:24.277 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:24.278 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:07:24.278 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:24.278 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 896 k 2024-09-15T00:07:24.278 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-09-15T00:07:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T00:07:24.278 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:24.278 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:24.279 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:24.279 INFO:teuthology.orchestra.run.smithi079.stdout:Install 2 Packages 2024-09-15T00:07:24.279 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:24.279 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 976 k 2024-09-15T00:07:24.279 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 2.9 M 2024-09-15T00:07:24.279 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T00:07:24.349 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T00:07:24.363 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:24.364 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:24.422 INFO:teuthology.orchestra.run.smithi079.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 672 kB/s | 80 kB 00:00 2024-09-15T00:07:24.597 INFO:teuthology.orchestra.run.smithi079.stdout:(2/2): ceph-fuse-19.1.1-324.gecab7be5.el9.x86_6 3.0 MB/s | 896 kB 00:00 2024-09-15T00:07:24.598 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:24.598 INFO:teuthology.orchestra.run.smithi079.stdout:Total 3.0 MB/s | 976 kB 00:00 2024-09-15T00:07:24.610 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:24.632 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:24.632 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:24.669 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:24.713 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:24.714 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T00:07:24.907 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T00:07:25.049 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T00:07:25.106 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T00:07:25.113 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:07:25.153 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T00:07:25.187 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T00:07:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T00:07:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:07:25.427 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T00:07:25.507 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T00:07:25.571 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T00:07:25.600 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/2 2024-09-15T00:07:26.007 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-15T00:07:26.007 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 1/13 2024-09-15T00:07:26.007 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T00:07:26.007 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T00:07:26.007 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T00:07:26.008 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T00:07:26.008 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T00:07:26.008 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T00:07:26.008 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T00:07:26.008 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T00:07:26.008 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T00:07:26.008 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T00:07:26.008 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T00:07:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T00:07:26.009 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:26.010 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:26.010 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T00:07:26.010 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:26.010 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:26.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-volume 2024-09-15T00:07:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T00:07:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:26.346 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T00:07:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T00:07:26.346 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:26.347 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:26.360 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:07:26.361 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/6 2024-09-15T00:07:26.361 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T00:07:26.361 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T00:07:26.361 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T00:07:26.361 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T00:07:26.592 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T00:07:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T00:07:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:26.635 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T00:07:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T00:07:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T00:07:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T00:07:26.635 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T00:07:26.636 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:26.636 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:26.887 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:32 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:26.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-fuse 2024-09-15T00:07:27.027 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:27.028 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:27.028 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-09-15T00:07:27.028 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:27.029 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 285 k 2024-09-15T00:07:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:27.029 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:27.029 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:27.029 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-15T00:07:27.029 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:27.029 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 285 k 2024-09-15T00:07:27.032 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 1.3 M 2024-09-15T00:07:27.032 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:27.183 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:26 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:27.279 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-volume-19.1.1-324.gecab7be5.el9.noarch.rpm 1.1 MB/s | 285 kB 00:00 2024-09-15T00:07:27.280 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:27.280 INFO:teuthology.orchestra.run.smithi079.stdout:Total 1.1 MB/s | 285 kB 00:00 2024-09-15T00:07:27.280 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:27.288 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:27.288 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:27.311 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:27.312 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:27.322 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:27.323 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:07:27.323 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:27.323 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 146 k 2024-09-15T00:07:27.323 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-15T00:07:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T00:07:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T00:07:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T00:07:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T00:07:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T00:07:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:27.324 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:27.324 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:27.324 INFO:teuthology.orchestra.run.smithi027.stdout:Install 6 Packages 2024-09-15T00:07:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:27.325 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 7.0 M 2024-09-15T00:07:27.325 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 33 M 2024-09-15T00:07:27.325 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:27.477 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:31 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:27.606 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:07:27.611 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:27.612 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:27.612 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:07:27.612 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:27.612 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:27.612 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 896 k 2024-09-15T00:07:27.612 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-15T00:07:27.612 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T00:07:27.612 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:27.612 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:27.613 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:27.613 INFO:teuthology.orchestra.run.smithi092.stdout:Install 2 Packages 2024-09-15T00:07:27.613 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:27.613 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 976 k 2024-09-15T00:07:27.613 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 2.9 M 2024-09-15T00:07:27.613 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:07:27.622 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:07:27.622 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T00:07:27.622 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:27.804 INFO:teuthology.orchestra.run.smithi027.stdout:(1/6): ceph-mgr-cephadm-19.1.1-324.gecab7be5.el 783 kB/s | 146 kB 00:00 2024-09-15T00:07:27.854 INFO:teuthology.orchestra.run.smithi027.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-15T00:07:27.879 INFO:teuthology.orchestra.run.smithi027.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-15T00:07:27.904 INFO:teuthology.orchestra.run.smithi027.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-15T00:07:28.004 INFO:teuthology.orchestra.run.smithi092.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 595 kB/s | 80 kB 00:00 2024-09-15T00:07:28.163 INFO:teuthology.orchestra.run.smithi092.stdout:(2/2): ceph-fuse-19.1.1-324.gecab7be5.el9.x86_6 3.0 MB/s | 896 kB 00:00 2024-09-15T00:07:28.163 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:28.164 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.7 MB/s | 976 kB 00:00 2024-09-15T00:07:28.176 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:28.198 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:28.199 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:28.255 INFO:teuthology.orchestra.run.smithi027.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 389 kB/s | 248 kB 00:00 2024-09-15T00:07:28.279 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:28.280 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:28.473 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:28.606 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T00:07:28.644 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:07:28.644 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:28.644 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:28.644 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:28.644 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:28.644 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:28.670 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:07:28.755 INFO:teuthology.orchestra.run.smithi027.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-15T00:07:28.757 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:28.757 INFO:teuthology.orchestra.run.smithi027.stdout:Total 4.9 MB/s | 7.0 MB 00:01 2024-09-15T00:07:28.834 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:28.847 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:28.847 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:28.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librados-devel 2024-09-15T00:07:28.954 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:28.956 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:29.271 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:29.347 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:07:29.347 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/2 2024-09-15T00:07:29.437 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T00:07:29.483 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T00:07:29.520 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:35 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:29.623 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T00:07:29.623 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:29.623 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:29.623 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T00:07:29.623 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:29.623 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:29.658 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:29.658 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:29.659 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2024-09-15T00:07:29.659 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:29.659 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:29.659 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 127 k 2024-09-15T00:07:29.659 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:29.659 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:29.659 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:29.659 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-15T00:07:29.659 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:29.660 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 127 k 2024-09-15T00:07:29.660 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 456 k 2024-09-15T00:07:29.660 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:29.870 INFO:teuthology.orchestra.run.smithi079.stdout:librados-devel-19.1.1-324.gecab7be5.el9.x86_64. 604 kB/s | 127 kB 00:00 2024-09-15T00:07:29.870 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:29.870 INFO:teuthology.orchestra.run.smithi079.stdout:Total 599 kB/s | 127 kB 00:00 2024-09-15T00:07:29.871 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:29.876 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:29.876 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:29.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-volume 2024-09-15T00:07:29.942 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:29.942 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:30.036 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T00:07:30.091 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:30.149 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T00:07:30.210 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T00:07:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:07:30.500 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:30.640 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:30.640 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 285 k 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 285 k 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 1.3 M 2024-09-15T00:07:30.641 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:30.644 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:30.867 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-volume-19.1.1-324.gecab7be5.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-15T00:07:30.868 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:30.868 INFO:teuthology.orchestra.run.smithi092.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-15T00:07:30.868 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:30.869 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:30.869 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:30.869 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:30.869 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:30.869 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:30.869 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:30.876 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:30.876 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:30.899 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:30.911 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:31.035 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-15T00:07:31.036 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/6 2024-09-15T00:07:31.036 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T00:07:31.036 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T00:07:31.036 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T00:07:31.036 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T00:07:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:31.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install libcephfs2 2024-09-15T00:07:31.186 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:07:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:07:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:07:31.201 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T00:07:31.201 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:31.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T00:07:31.302 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:31.302 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:31.302 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:31.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T00:07:31.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T00:07:31.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T00:07:31.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T00:07:31.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T00:07:31.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:31.303 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:31.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-fuse 2024-09-15T00:07:31.710 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:31.804 INFO:teuthology.orchestra.run.smithi079.stdout:Package libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:31.849 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:31.850 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:07:31.850 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:31.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install libcephfs-devel 2024-09-15T00:07:32.153 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:31 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:32.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:07:32.167 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:32.167 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:32.167 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:32.167 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:32.168 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:32.295 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:32.295 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:32.295 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:07:32.295 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:32.295 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:32.296 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 896 k 2024-09-15T00:07:32.296 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-15T00:07:32.296 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T00:07:32.296 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:32.296 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:32.296 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:32.296 INFO:teuthology.orchestra.run.smithi027.stdout:Install 2 Packages 2024-09-15T00:07:32.296 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:32.297 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 976 k 2024-09-15T00:07:32.297 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 2.9 M 2024-09-15T00:07:32.297 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:32.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados-devel 2024-09-15T00:07:32.483 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:38 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:32.616 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:32.617 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:32.617 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2024-09-15T00:07:32.617 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:32.617 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:32.617 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 32 k 2024-09-15T00:07:32.617 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:32.618 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:32.618 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:32.618 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-15T00:07:32.618 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:32.618 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 32 k 2024-09-15T00:07:32.618 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 142 k 2024-09-15T00:07:32.618 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:32.674 INFO:teuthology.orchestra.run.smithi027.stdout:(1/2): ceph-fuse-19.1.1-324.gecab7be5.el9.x86_6 2.7 MB/s | 896 kB 00:00 2024-09-15T00:07:32.851 INFO:teuthology.orchestra.run.smithi079.stdout:libcephfs-devel-19.1.1-324.gecab7be5.el9.x86_64 136 kB/s | 32 kB 00:00 2024-09-15T00:07:32.851 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:32.851 INFO:teuthology.orchestra.run.smithi079.stdout:Total 135 kB/s | 32 kB 00:00 2024-09-15T00:07:32.852 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:32.855 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:32.855 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:32.858 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:32.859 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:32.944 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:33.016 INFO:teuthology.orchestra.run.smithi027.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 119 kB/s | 80 kB 00:00 2024-09-15T00:07:33.016 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:33.017 INFO:teuthology.orchestra.run.smithi027.stdout:Total 1.3 MB/s | 976 kB 00:00 2024-09-15T00:07:33.029 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:33.033 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:33.039 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:33.054 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:33.054 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:33.142 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:33.142 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:33.173 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:33.173 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-15T00:07:33.173 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:33.173 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 127 k 2024-09-15T00:07:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-15T00:07:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 127 k 2024-09-15T00:07:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 456 k 2024-09-15T00:07:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:33.339 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:33.363 INFO:teuthology.orchestra.run.smithi092.stdout:librados-devel-19.1.1-324.gecab7be5.el9.x86_64. 663 kB/s | 127 kB 00:00 2024-09-15T00:07:33.364 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:33.364 INFO:teuthology.orchestra.run.smithi092.stdout:Total 657 kB/s | 127 kB 00:00 2024-09-15T00:07:33.364 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:33.369 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:33.370 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:33.433 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:33.434 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:33.441 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:33.480 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T00:07:33.551 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:07:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:33.667 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:33.667 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:33.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librados2 2024-09-15T00:07:34.119 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-15T00:07:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/2 2024-09-15T00:07:34.370 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:34.370 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:34.370 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:34.370 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:34.371 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:34.371 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:34.464 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T00:07:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:34.511 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T00:07:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:34.511 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:34.554 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:34.597 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:34.598 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:07:34.598 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:34.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs2 2024-09-15T00:07:34.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librbd1 2024-09-15T00:07:34.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-volume 2024-09-15T00:07:35.224 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:35.319 INFO:teuthology.orchestra.run.smithi092.stdout:Package libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:35.322 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:35.364 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:35.365 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:07:35.365 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:35.366 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:35.367 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:07:35.367 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:35.403 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:35.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install libcephfs-devel 2024-09-15T00:07:35.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rados 2024-09-15T00:07:35.544 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:35.544 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 285 k 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 285 k 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 1.3 M 2024-09-15T00:07:35.545 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:36.004 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:36.009 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:36.037 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-volume-19.1.1-324.gecab7be5.el9.noarch.rpm 581 kB/s | 285 kB 00:00 2024-09-15T00:07:36.037 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:36.037 INFO:teuthology.orchestra.run.smithi027.stdout:Total 579 kB/s | 285 kB 00:00 2024-09-15T00:07:36.038 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:36.047 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:36.047 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:36.073 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:36.074 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:36.100 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:36.138 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:36.139 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 32 k 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 32 k 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 142 k 2024-09-15T00:07:36.140 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:36.143 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:36.144 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:07:36.144 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:36.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rgw 2024-09-15T00:07:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:07:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:07:36.389 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:07:36.389 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T00:07:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:36.391 INFO:teuthology.orchestra.run.smithi092.stdout:libcephfs-devel-19.1.1-324.gecab7be5.el9.x86_64 126 kB/s | 32 kB 00:00 2024-09-15T00:07:36.391 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:36.392 INFO:teuthology.orchestra.run.smithi092.stdout:Total 126 kB/s | 32 kB 00:00 2024-09-15T00:07:36.392 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:36.395 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:36.396 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:36.400 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:36.400 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:36.493 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:36.588 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:36.783 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:36.874 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:36.918 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:36.919 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:07:36.919 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:36.964 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:36.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-cephfs 2024-09-15T00:07:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:37.190 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:37.191 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:37.191 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:37.191 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:37.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librados2 2024-09-15T00:07:37.406 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-15T00:07:37.407 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:37.407 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:37.407 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-15T00:07:37.407 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:37.407 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:37.554 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:43 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:37.643 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:37.687 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:37.688 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:07:37.688 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:37.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librados-devel 2024-09-15T00:07:37.768 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rbd 2024-09-15T00:07:37.947 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:38.039 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:38.085 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:38.086 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:07:38.086 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:38.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install librbd1 2024-09-15T00:07:38.296 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:38.335 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:44 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:38.426 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:38.437 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:38.437 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:38.437 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-09-15T00:07:38.437 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:38.437 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:38.437 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 127 k 2024-09-15T00:07:38.437 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:38.437 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:38.437 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:38.438 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-15T00:07:38.438 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:38.438 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 127 k 2024-09-15T00:07:38.438 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 456 k 2024-09-15T00:07:38.438 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:38.470 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:38.471 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:07:38.471 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:38.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-fuse 2024-09-15T00:07:38.641 INFO:teuthology.orchestra.run.smithi027.stdout:librados-devel-19.1.1-324.gecab7be5.el9.x86_64. 618 kB/s | 127 kB 00:00 2024-09-15T00:07:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:Total 614 kB/s | 127 kB 00:00 2024-09-15T00:07:38.642 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:38.648 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:38.715 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:38.716 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:38.719 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:38.809 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:38.853 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:38.854 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:07:38.854 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:38.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rados 2024-09-15T00:07:38.976 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:39.129 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:45 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:39.263 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:39.264 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:39.264 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-09-15T00:07:39.264 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:39.264 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:39.264 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 85 k 2024-09-15T00:07:39.264 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:39.264 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:39.265 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:39.265 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-15T00:07:39.265 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:39.265 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 85 k 2024-09-15T00:07:39.265 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 231 k 2024-09-15T00:07:39.265 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:39.401 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-fuse-19.1.1-324.gecab7be5.el9.x86_64.rpm 431 kB/s | 85 kB 00:00 2024-09-15T00:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout:Total 428 kB/s | 85 kB 00:00 2024-09-15T00:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:39.494 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:43 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:39.585 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:39.627 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:39.628 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:07:39.629 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:39.659 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:39.659 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:39.659 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:39.659 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:39.659 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:39.659 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:39.692 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rgw 2024-09-15T00:07:39.692 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:39.793 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:39.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs2 2024-09-15T00:07:40.151 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:40.263 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:44 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:40.353 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:40.396 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:40.397 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:07:40.397 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:40.398 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:40.398 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:40.398 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:40.398 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:40.398 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:40.398 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:40.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-cephfs 2024-09-15T00:07:40.491 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:40.585 INFO:teuthology.orchestra.run.smithi027.stdout:Package libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:40.631 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:40.632 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:07:40.632 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:40.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs-devel 2024-09-15T00:07:40.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-mirror 2024-09-15T00:07:41.046 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:45 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:41.137 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:41.180 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:41.181 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:07:41.181 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:41.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install python3-rbd 2024-09-15T00:07:41.277 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:47 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:41.289 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:41.416 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:41.416 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:41.416 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2024-09-15T00:07:41.416 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:41.416 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:41.417 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.1 M 2024-09-15T00:07:41.417 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:41.417 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:41.417 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:41.417 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-15T00:07:41.417 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:41.417 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 3.1 M 2024-09-15T00:07:41.418 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 13 M 2024-09-15T00:07:41.418 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:41.429 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:41.429 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:41.429 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-09-15T00:07:41.429 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:41.429 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:41.429 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 32 k 2024-09-15T00:07:41.430 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:41.430 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:41.430 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:41.430 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-15T00:07:41.430 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:41.430 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 32 k 2024-09-15T00:07:41.431 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 142 k 2024-09-15T00:07:41.431 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:41.587 INFO:teuthology.orchestra.run.smithi027.stdout:libcephfs-devel-19.1.1-324.gecab7be5.el9.x86_64 203 kB/s | 32 kB 00:00 2024-09-15T00:07:41.588 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:41.588 INFO:teuthology.orchestra.run.smithi027.stdout:Total 201 kB/s | 32 kB 00:00 2024-09-15T00:07:41.588 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:41.591 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:41.592 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:41.595 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:41.596 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:41.686 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:41.768 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-mirror-19.1.1-324.gecab7be5.el9.x86_64.rpm 8.8 MB/s | 3.1 MB 00:00 2024-09-15T00:07:41.769 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:41.770 INFO:teuthology.orchestra.run.smithi079.stdout:Total 8.8 MB/s | 3.1 MB 00:00 2024-09-15T00:07:41.770 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:41.781 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:41.781 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:41.796 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:41.827 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:45 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:41.859 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:41.859 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:41.918 INFO:teuthology.orchestra.run.smithi092.stdout:Package python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:41.960 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:41.961 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:07:41.962 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:42.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-fuse 2024-09-15T00:07:42.153 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:42.302 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:42.383 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:42.383 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:42.383 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:42.397 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:07:42.397 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T00:07:42.397 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:07:42.397 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:07:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:42.577 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librados2 2024-09-15T00:07:42.603 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:42.737 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:42.737 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:42.737 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-15T00:07:42.737 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 85 k 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 85 k 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 231 k 2024-09-15T00:07:42.738 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:42.935 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-fuse-19.1.1-324.gecab7be5.el9.x86_64.rpm 433 kB/s | 85 kB 00:00 2024-09-15T00:07:42.936 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:42.936 INFO:teuthology.orchestra.run.smithi092.stdout:Total 430 kB/s | 85 kB 00:00 2024-09-15T00:07:42.936 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:42.943 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:42.943 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:43.009 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:43.009 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:43.166 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:43.169 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:43.264 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:43.283 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:43.311 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:43.312 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:07:43.312 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:43.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librbd1 2024-09-15T00:07:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:43.456 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:43.456 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:43.457 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:43.457 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:43.697 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:43.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-nbd 2024-09-15T00:07:43.904 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:43.905 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:43.905 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:43.905 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:43.961 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:44.057 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:44.104 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:44.105 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:07:44.105 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:44.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-mirror 2024-09-15T00:07:44.168 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rados 2024-09-15T00:07:44.297 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:50 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:44.436 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:44.436 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:44.436 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-09-15T00:07:44.436 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:44.436 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-09-15T00:07:44.436 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 170 k 2024-09-15T00:07:44.437 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:44.437 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-09-15T00:07:44.437 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-09-15T00:07:44.437 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2024-09-15T00:07:44.437 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:44.437 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 170 k 2024-09-15T00:07:44.437 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 494 k 2024-09-15T00:07:44.437 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-09-15T00:07:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-nbd-19.1.1-324.gecab7be5.el9.x86_64.rpm 811 kB/s | 170 kB 00:00 2024-09-15T00:07:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:44.647 INFO:teuthology.orchestra.run.smithi079.stdout:Total 804 kB/s | 170 kB 00:00 2024-09-15T00:07:44.648 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-09-15T00:07:44.655 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-09-15T00:07:44.655 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-09-15T00:07:44.719 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-09-15T00:07:44.719 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-09-15T00:07:44.745 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:48 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:44.764 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:43 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:44.858 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:44.874 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-09-15T00:07:44.886 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:44.886 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2024-09-15T00:07:44.886 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:44.886 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.1 M 2024-09-15T00:07:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:44.887 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:44.887 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:44.887 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-15T00:07:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:44.888 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 3.1 M 2024-09-15T00:07:44.888 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 13 M 2024-09-15T00:07:44.888 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:44.903 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:44.905 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:07:44.905 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:44.958 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rgw 2024-09-15T00:07:44.969 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:45.230 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-mirror-19.1.1-324.gecab7be5.el9.x86_64.rpm 9.1 MB/s | 3.1 MB 00:00 2024-09-15T00:07:45.231 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:45.232 INFO:teuthology.orchestra.run.smithi092.stdout:Total 9.0 MB/s | 3.1 MB 00:00 2024-09-15T00:07:45.232 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:45.246 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:45.246 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:45.369 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:45.370 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:45.552 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:44 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:45.630 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-09-15T00:07:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:45.630 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:45.646 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:45.693 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:45.694 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:07:45.694 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:45.750 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-cephfs 2024-09-15T00:07:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:45.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-nbd 2024-09-15T00:07:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:45.931 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:07:45.932 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T00:07:45.932 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:07:45.932 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:07:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:46.342 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:45 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:46.435 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:46.481 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:46.482 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:07:46.482 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:46.488 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:52 ago on Sun 15 Sep 2024 12:05:54 AM UTC. 2024-09-15T00:07:46.546 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rbd 2024-09-15T00:07:46.580 INFO:teuthology.orchestra.run.smithi079.stdout:Package rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:46.624 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-09-15T00:07:46.626 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-09-15T00:07:46.626 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-09-15T00:07:46.709 DEBUG:teuthology.parallel:result is None 2024-09-15T00:07:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:46.972 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:46.972 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:46.973 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:46.973 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:47.135 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:47.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-nbd 2024-09-15T00:07:47.229 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:47.276 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:47.277 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:07:47.277 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:47.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-fuse 2024-09-15T00:07:47.800 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:51 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:47.925 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:47.937 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:47.937 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-15T00:07:47.938 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:47.938 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-15T00:07:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 170 k 2024-09-15T00:07:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:47.938 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-15T00:07:47.938 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-15T00:07:47.938 INFO:teuthology.orchestra.run.smithi092.stdout:Install 1 Package 2024-09-15T00:07:47.938 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:47.939 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 170 k 2024-09-15T00:07:47.939 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 494 k 2024-09-15T00:07:47.939 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-15T00:07:48.065 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:48.066 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:48.066 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-15T00:07:48.066 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:48.066 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:48.066 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 85 k 2024-09-15T00:07:48.066 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:48.066 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:48.067 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:48.067 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-15T00:07:48.067 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:48.067 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 85 k 2024-09-15T00:07:48.067 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 231 k 2024-09-15T00:07:48.067 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:48.124 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-nbd-19.1.1-324.gecab7be5.el9.x86_64.rpm 921 kB/s | 170 kB 00:00 2024-09-15T00:07:48.125 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:48.125 INFO:teuthology.orchestra.run.smithi092.stdout:Total 912 kB/s | 170 kB 00:00 2024-09-15T00:07:48.125 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-15T00:07:48.133 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-15T00:07:48.133 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-15T00:07:48.196 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-15T00:07:48.196 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-15T00:07:48.256 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-fuse-19.1.1-324.gecab7be5.el9.x86_64.rpm 451 kB/s | 85 kB 00:00 2024-09-15T00:07:48.257 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:48.257 INFO:teuthology.orchestra.run.smithi027.stdout:Total 447 kB/s | 85 kB 00:00 2024-09-15T00:07:48.257 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:48.265 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:48.265 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:48.334 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:48.334 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-15T00:07:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:48.616 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:48.886 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:49.006 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:49.132 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:49.133 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-15T00:07:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:49.133 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:49.134 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:49.243 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:49.243 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:49.243 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:49.243 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:49.244 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:49.244 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:49.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install rbd-nbd 2024-09-15T00:07:49.505 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-mirror 2024-09-15T00:07:49.943 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:53 ago on Sun 15 Sep 2024 12:05:56 AM UTC. 2024-09-15T00:07:50.035 INFO:teuthology.orchestra.run.smithi092.stdout:Package rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:50.079 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-15T00:07:50.080 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2024-09-15T00:07:50.081 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-15T00:07:50.114 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:49 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:50.163 DEBUG:teuthology.parallel:result is None 2024-09-15T00:07:50.255 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:50.256 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:50.256 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-09-15T00:07:50.257 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:50.257 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:50.257 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.1 M 2024-09-15T00:07:50.257 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:50.257 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:50.257 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:50.258 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-15T00:07:50.258 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:50.258 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 3.1 M 2024-09-15T00:07:50.258 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 13 M 2024-09-15T00:07:50.258 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:50.603 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-mirror-19.1.1-324.gecab7be5.el9.x86_64.rpm 9.0 MB/s | 3.1 MB 00:00 2024-09-15T00:07:50.603 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:50.603 INFO:teuthology.orchestra.run.smithi027.stdout:Total 8.9 MB/s | 3.1 MB 00:00 2024-09-15T00:07:50.604 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:50.614 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:50.614 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:50.699 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:50.700 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:51.154 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:51.203 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:51.230 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:51.231 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T00:07:51.231 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T00:07:51.231 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:07:51.231 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T00:07:51.231 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:52.313 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:52.313 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:52.313 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:52.313 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:52.314 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:52.314 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:52.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-nbd 2024-09-15T00:07:53.138 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:52 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:53.279 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:53.279 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-15T00:07:53.279 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:53.280 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-15T00:07:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 170 k 2024-09-15T00:07:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:53.280 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-15T00:07:53.280 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-15T00:07:53.280 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-09-15T00:07:53.281 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:53.281 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 170 k 2024-09-15T00:07:53.282 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 494 k 2024-09-15T00:07:53.282 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-15T00:07:53.473 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-nbd-19.1.1-324.gecab7be5.el9.x86_64.rpm 886 kB/s | 170 kB 00:00 2024-09-15T00:07:53.473 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-15T00:07:53.474 INFO:teuthology.orchestra.run.smithi027.stdout:Total 881 kB/s | 170 kB 00:00 2024-09-15T00:07:53.474 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-15T00:07:53.482 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-15T00:07:53.482 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-15T00:07:53.549 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-15T00:07:53.549 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-15T00:07:53.711 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-15T00:07:53.815 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:54.223 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:54.475 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-15T00:07:54.475 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:54.475 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-15T00:07:54.475 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-15T00:07:54.475 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:54.475 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:54.691 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-nbd 2024-09-15T00:07:55.285 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:54 ago on Sun 15 Sep 2024 12:06:01 AM UTC. 2024-09-15T00:07:55.383 INFO:teuthology.orchestra.run.smithi027.stdout:Package rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-15T00:07:55.432 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-15T00:07:55.433 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-09-15T00:07:55.433 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-15T00:07:55.493 DEBUG:teuthology.parallel:result is None 2024-09-15T00:07:55.494 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:07:55.641 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T00:07:55.663 INFO:teuthology.orchestra.run.smithi027.stdout:19.1.1-324.gecab7be5.el9 2024-09-15T00:07:55.664 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324.gecab7be5.el9 2024-09-15T00:07:55.664 INFO:teuthology.task.install:The correct ceph version 19.1.1-324.gecab7be5 is installed. 2024-09-15T00:07:55.666 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:07:55.806 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T00:07:55.831 INFO:teuthology.orchestra.run.smithi079.stdout:19.1.1-324.gecab7be5.el9 2024-09-15T00:07:55.831 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324.gecab7be5.el9 2024-09-15T00:07:55.831 INFO:teuthology.task.install:The correct ceph version 19.1.1-324.gecab7be5 is installed. 2024-09-15T00:07:55.833 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-15T00:07:55.974 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T00:07:55.997 INFO:teuthology.orchestra.run.smithi092.stdout:19.1.1-324.gecab7be5.el9 2024-09-15T00:07:55.997 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324.gecab7be5.el9 2024-09-15T00:07:55.997 INFO:teuthology.task.install:The correct ceph version 19.1.1-324.gecab7be5 is installed. 2024-09-15T00:07:55.999 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T00:07:55.999 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:55.999 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T00:07:56.026 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:07:56.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T00:07:56.055 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T00:07:56.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T00:07:56.081 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T00:07:56.082 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:56.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T00:07:56.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T00:07:56.174 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:07:56.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T00:07:56.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T00:07:56.272 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T00:07:56.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T00:07:56.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T00:07:56.364 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T00:07:56.365 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:56.365 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T00:07:56.391 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T00:07:56.457 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:07:56.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T00:07:56.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T00:07:56.556 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T00:07:56.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T00:07:56.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T00:07:56.648 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T00:07:56.648 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:56.648 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T00:07:56.674 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T00:07:56.740 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:07:56.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T00:07:56.769 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T00:07:56.838 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T00:07:56.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T00:07:56.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T00:07:56.930 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T00:07:57.034 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T00:07:57.034 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /var/log/ceph 2024-09-15T00:07:57.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /var/log/ceph 2024-09-15T00:07:57.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2024-09-15T00:07:57.067 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T00:07:57.067 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T00:07:57.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T00:07:57.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T00:07:57.137 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T00:07:57.138 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T00:07:57.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T00:07:57.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T00:07:57.214 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T00:07:57.214 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T00:07:57.215 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7904812', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 4271, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'sleep_before_teardown': 0, 'subset': '97/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'targets': {'smithi027.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMaaqULEJUBl8gf4QfIOoAkM81E2NSuXrn1aVFZDTd/CMyYmw5QazkTn/4bFBur/xubsieQcD3qsxFTFgd0RC/o=', 'smithi079.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHP03GaIXMYrkiGBHT2/QmEQ7JGBgKl8K3PITsZt30dxNGeTV8nuSXq9OaRMalF/28Mimtbs1amUL1E6ndFqtzE=', 'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIkrFPLjFX7ySN3YxB2gqiu5JhWWYXuImBe6VXTcw6Obgu19Imj4fTZvpVecUAlX+pxMOxbNazVZyxR0nnu+mUQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-14_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T00:07:57.215 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T00:07:57.238 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T00:07:57.248 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T00:07:57.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T00:07:57.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T00:07:57.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T00:07:57.332 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:57.332 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T00:07:57.375 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T00:07:57.375 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-09-15T00:07:57.431 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T00:07:57.431 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:07:57.431 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T00:07:57.431 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:07:57.431 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:07:57.431 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-15 00:07:51.481441740 +0000 2024-09-15T00:07:57.431 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-15 00:05:17.597372464 +0000 2024-09-15T00:07:57.431 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-15 00:05:17.597372464 +0000 2024-09-15T00:07:57.432 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-15 00:05:17.597372464 +0000 2024-09-15T00:07:57.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T00:07:57.495 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-15T00:07:57.495 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-15T00:07:57.495 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000255086 s, 2.0 MB/s 2024-09-15T00:07:57.497 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T00:07:57.554 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-09-15T00:07:57.609 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T00:07:57.610 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:07:57.610 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T00:07:57.610 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:07:57.610 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:07:57.610 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-15 00:07:51.482441714 +0000 2024-09-15T00:07:57.610 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-15 00:05:17.938363605 +0000 2024-09-15T00:07:57.610 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-15 00:05:17.938363605 +0000 2024-09-15T00:07:57.610 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-15 00:05:17.938363605 +0000 2024-09-15T00:07:57.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T00:07:57.673 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-15T00:07:57.674 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-15T00:07:57.674 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000207444 s, 2.5 MB/s 2024-09-15T00:07:57.675 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T00:07:57.731 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-09-15T00:07:57.787 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T00:07:57.787 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:07:57.787 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-15T00:07:57.787 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:07:57.787 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:07:57.787 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-15 00:07:51.482441714 +0000 2024-09-15T00:07:57.787 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-15 00:05:18.273354902 +0000 2024-09-15T00:07:57.787 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-15 00:05:18.273354902 +0000 2024-09-15T00:07:57.787 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-15 00:05:18.273354902 +0000 2024-09-15T00:07:57.787 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T00:07:57.851 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-15T00:07:57.851 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-15T00:07:57.851 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000211217 s, 2.4 MB/s 2024-09-15T00:07:57.852 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T00:07:57.910 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-09-15T00:07:57.966 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T00:07:57.966 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:07:57.966 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-15T00:07:57.967 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:07:57.967 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:07:57.967 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-15 00:07:51.483441689 +0000 2024-09-15T00:07:57.967 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-15 00:05:18.613346069 +0000 2024-09-15T00:07:57.967 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-15 00:05:18.613346069 +0000 2024-09-15T00:07:57.967 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-15 00:05:18.613346069 +0000 2024-09-15T00:07:57.968 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T00:07:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-15T00:07:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-15T00:07:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000188472 s, 2.7 MB/s 2024-09-15T00:07:58.031 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T00:07:58.088 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T00:07:58.088 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:07:58.088 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T00:07:58.107 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T00:07:58.107 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-09-15T00:07:58.165 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T00:07:58.166 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:07:58.166 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-15T00:07:58.166 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:07:58.166 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:07:58.166 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-15 00:07:42.655815997 +0000 2024-09-15T00:07:58.166 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-15 00:05:14.643377569 +0000 2024-09-15T00:07:58.166 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-15 00:05:14.643377569 +0000 2024-09-15T00:07:58.166 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-15 00:05:14.643377569 +0000 2024-09-15T00:07:58.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T00:07:58.232 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-15T00:07:58.232 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-15T00:07:58.232 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000222214 s, 2.3 MB/s 2024-09-15T00:07:58.233 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T00:07:58.291 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-09-15T00:07:58.349 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T00:07:58.349 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:07:58.349 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T00:07:58.349 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:07:58.349 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:07:58.349 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-15 00:07:42.655815997 +0000 2024-09-15T00:07:58.349 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-15 00:05:14.978369760 +0000 2024-09-15T00:07:58.349 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-15 00:05:14.978369760 +0000 2024-09-15T00:07:58.349 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-15 00:05:14.978369760 +0000 2024-09-15T00:07:58.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T00:07:58.415 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-15T00:07:58.415 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-15T00:07:58.415 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000195995 s, 2.6 MB/s 2024-09-15T00:07:58.416 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T00:07:58.474 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-09-15T00:07:58.531 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T00:07:58.531 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:07:58.531 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T00:07:58.531 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:07:58.531 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:07:58.531 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-15 00:07:42.656815973 +0000 2024-09-15T00:07:58.531 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-15 00:05:15.291362465 +0000 2024-09-15T00:07:58.531 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-15 00:05:15.291362465 +0000 2024-09-15T00:07:58.532 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-15 00:05:15.291362465 +0000 2024-09-15T00:07:58.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T00:07:58.596 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-15T00:07:58.596 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-15T00:07:58.596 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000268529 s, 1.9 MB/s 2024-09-15T00:07:58.598 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T00:07:58.654 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-09-15T00:07:58.710 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T00:07:58.710 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T00:07:58.710 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-15T00:07:58.710 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T00:07:58.711 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T00:07:58.711 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-15 00:07:42.656815973 +0000 2024-09-15T00:07:58.711 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-15 00:05:15.618354842 +0000 2024-09-15T00:07:58.711 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-15 00:05:15.618354842 +0000 2024-09-15T00:07:58.711 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-09-15 00:05:15.618354842 +0000 2024-09-15T00:07:58.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T00:07:58.777 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-15T00:07:58.777 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-15T00:07:58.777 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000290305 s, 1.8 MB/s 2024-09-15T00:07:58.777 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T00:07:58.833 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T00:07:58.833 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-15T00:07:58.833 INFO:tasks.ceph:Generating config... 2024-09-15T00:07:58.835 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T00:07:58.835 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-15T00:07:58.835 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-15T00:07:58.835 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T00:07:58.835 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T00:07:58.835 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T00:07:58.835 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T00:07:58.835 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T00:07:58.835 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T00:07:58.835 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-15T00:07:58.836 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T00:07:58.837 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-15T00:07:58.837 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T00:07:58.837 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T00:07:58.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T00:07:58.895 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T00:07:58.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T00:07:58.945 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T00:07:58.970 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.27'), ('mon.c', '[v2:172.21.15.27:3301,v1:172.21.15.27:6790]'), ('mon.b', '172.21.15.79')] 2024-09-15T00:07:58.971 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.27,[v2:172.21.15.27:3301,v1:172.21.15.27:6790],172.21.15.79', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-15T00:07:58.972 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:58.972 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T00:07:59.027 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.27 --addv c '[v2:172.21.15.27:3301,v1:172.21.15.27:6790]' --add b 172.21.15.79 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:07:59.114 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:07:59.114 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: generated fsid 4f532d31-28c1-4236-a74e-6392960baf2b 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 0 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 4f532d31-28c1-4236-a74e-6392960baf2b 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:last_changed 2024-09-15T00:07:59.113803+0000 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-15T00:07:59.113803+0000 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:min_mon_release 17 (quincy) 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:election_strategy: 1 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.b 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:2: [v2:172.21.15.27:3301/0,v1:172.21.15.27:6790/0] mon.c 2024-09-15T00:07:59.115 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T00:07:59.117 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T00:07:59.133 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4f532d31-28c1-4236-a74e-6392960baf2b... 2024-09-15T00:07:59.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T00:07:59.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T00:07:59.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T00:07:59.218 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-09-15T00:07:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: chdir = "" 2024-09-15T00:07:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T00:07:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: auth supported = cephx 2024-09-15T00:07:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: filestore xattr use omap = true 2024-09-15T00:07:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: mon clock drift allowed = 1.000 2024-09-15T00:07:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: osd crush chooseleaf type = 0 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: auth debug = true 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on old message = true 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on bug = true 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn max object skew = 0 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default size = 2 2024-09-15T00:07:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow primary affinity = true 2024-09-15T00:07:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow pg remap = true 2024-09-15T00:07:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on legacy crush tunables = false 2024-09-15T00:07:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on crush straw calc version zero = false 2024-09-15T00:07:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on no sortbitwise = false 2024-09-15T00:07:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = false 2024-09-15T00:07:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on too few osds = false 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: mon_allow_pool_size_one = true 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: osd default data pool replay window = 5 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: mon allow pool delete = true 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: mon cluster log file level = debug 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: debug asserts on shutdown = true 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: mon health detail to clog = false 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: mon host = "172.21.15.27,[v2:172.21.15.27:3301,v1:172.21.15.27:6790],172.21.15.79" 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: mon client directed command retry = 5 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on pool no app = False 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject socket failures = 5000 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 4f532d31-28c1-4236-a74e-6392960baf2b 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout:[osd] 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: osd journal size = 100 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub load threshold = 5.0 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub max interval = 600 2024-09-15T00:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock profile = high_recovery_ops 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: osd recover clone overlap = true 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: osd recovery max chunk = 1048576 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug shutdown = true 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug op order = true 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify stray on activate = true 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug trim objects = true 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: osd open classes on start = true 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug pg log writeout = true 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: osd deep scrub update digest min age = 30 2024-09-15T00:07:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: osd map max advance = 10 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: journal zero on create = true 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: filestore ondisk finisher threads = 3 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: filestore apply finisher threads = 3 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: bdev debug aio = true 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug misdirected ops = true 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore block size = 96636764160 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore compression algorithm = lz4 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore compression mode = aggressive 2024-09-15T00:07:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore fsck on mount = True 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluefs = 1/20 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluestore = 1/20 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: debug osd = 20 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: debug rocksdb = 4/10 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.9 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.8 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: osd blocked scrub grace period = 3600 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: osd failsafe full ratio = 0.95 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: osd max markdown count = 1000 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: osd objectstore = bluestore 2024-09-15T00:07:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr] 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: debug mgr = 20 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: mgr/telemetry/nag = false 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout:[mon] 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-09-15T00:07:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: debug paxos = 20 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: mon data avail warn = 5 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: mon mgr mkfs grace = 240 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd reporter subtree level = osd 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd prime pg temp = true 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T00:07:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: # 1m isn't quite enough 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: mon_down_mkfs_grace = 2m 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_filestore_osds = false 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout:[client] 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: rgw cache enabled = true 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable ops log = true 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable usage log = true 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T00:07:59.228 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.a] 2024-09-15T00:07:59.229 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.c] 2024-09-15T00:07:59.229 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.b] 2024-09-15T00:07:59.229 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-15T00:07:59.229 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.230 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-09-15T00:07:59.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-09-15T00:07:59.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.27,[v2:172.21.15.27:3301,v1:172.21.15.27:6790],172.21.15.79" 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: mon client directed command retry = 5 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on pool no app = False 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject socket failures = 5000 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 4f532d31-28c1-4236-a74e-6392960baf2b 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-09-15T00:07:59.233 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-09-15T00:07:59.234 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-09-15T00:07:59.235 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore compression algorithm = lz4 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore compression mode = aggressive 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: osd blocked scrub grace period = 3600 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-09-15T00:07:59.236 INFO:teuthology.orchestra.run.smithi092.stdout: osd max markdown count = 1000 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-15T00:07:59.237 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.238 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T00:07:59.239 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi079.stdout: chdir = "" 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi079.stdout: auth supported = cephx 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi079.stdout: filestore xattr use omap = true 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi079.stdout: mon clock drift allowed = 1.000 2024-09-15T00:07:59.240 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.241 INFO:teuthology.orchestra.run.smithi079.stdout: osd crush chooseleaf type = 0 2024-09-15T00:07:59.241 INFO:teuthology.orchestra.run.smithi079.stdout: auth debug = true 2024-09-15T00:07:59.241 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.241 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on old message = true 2024-09-15T00:07:59.241 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on bug = true 2024-09-15T00:07:59.241 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.241 INFO:teuthology.orchestra.run.smithi079.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T00:07:59.241 INFO:teuthology.orchestra.run.smithi079.stdout: mon pg warn max object skew = 0 2024-09-15T00:07:59.241 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.242 INFO:teuthology.orchestra.run.smithi079.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T00:07:59.242 INFO:teuthology.orchestra.run.smithi079.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T00:07:59.242 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.242 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default size = 2 2024-09-15T00:07:59.242 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.242 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow primary affinity = true 2024-09-15T00:07:59.242 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow pg remap = true 2024-09-15T00:07:59.242 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on legacy crush tunables = false 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on crush straw calc version zero = false 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on no sortbitwise = false 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on osd down out interval zero = false 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on too few osds = false 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: mon_allow_pool_size_one = true 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.243 INFO:teuthology.orchestra.run.smithi079.stdout: osd default data pool replay window = 5 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: mon allow pool delete = true 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: mon cluster log file level = debug 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: debug asserts on shutdown = true 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: mon health detail to clog = false 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: mon host = "172.21.15.27,[v2:172.21.15.27:3301,v1:172.21.15.27:6790],172.21.15.79" 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: mon client directed command retry = 5 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on pool no app = False 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: ms inject socket failures = 5000 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = 4f532d31-28c1-4236-a74e-6392960baf2b 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.244 INFO:teuthology.orchestra.run.smithi079.stdout:[osd] 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: osd journal size = 100 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub load threshold = 5.0 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub max interval = 600 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: osd mclock profile = high_recovery_ops 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: osd recover clone overlap = true 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: osd recovery max chunk = 1048576 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug shutdown = true 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug op order = true 2024-09-15T00:07:59.245 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify stray on activate = true 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug trim objects = true 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: osd open classes on start = true 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug pg log writeout = true 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: osd deep scrub update digest min age = 30 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: osd map max advance = 10 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: journal zero on create = true 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: filestore ondisk finisher threads = 3 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: filestore apply finisher threads = 3 2024-09-15T00:07:59.246 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: bdev debug aio = true 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug misdirected ops = true 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore block size = 96636764160 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore compression algorithm = lz4 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore compression mode = aggressive 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore fsck on mount = True 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluefs = 1/20 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluestore = 1/20 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: debug osd = 20 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: debug rocksdb = 4/10 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd full ratio = 0.9 2024-09-15T00:07:59.247 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd nearfull ratio = 0.8 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: osd blocked scrub grace period = 3600 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: osd failsafe full ratio = 0.95 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: osd max markdown count = 1000 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: osd objectstore = bluestore 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr] 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: debug mgr = 20 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: mgr/telemetry/nag = false 2024-09-15T00:07:59.248 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.249 INFO:teuthology.orchestra.run.smithi079.stdout:[mon] 2024-09-15T00:07:59.249 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-09-15T00:07:59.249 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2024-09-15T00:07:59.249 INFO:teuthology.orchestra.run.smithi079.stdout: debug paxos = 20 2024-09-15T00:07:59.249 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2024-09-15T00:07:59.249 INFO:teuthology.orchestra.run.smithi079.stdout: mon data avail warn = 5 2024-09-15T00:07:59.249 INFO:teuthology.orchestra.run.smithi079.stdout: mon mgr mkfs grace = 240 2024-09-15T00:07:59.250 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2024-09-15T00:07:59.250 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd reporter subtree level = osd 2024-09-15T00:07:59.250 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd prime pg temp = true 2024-09-15T00:07:59.250 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2024-09-15T00:07:59.250 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.250 INFO:teuthology.orchestra.run.smithi079.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T00:07:59.250 INFO:teuthology.orchestra.run.smithi079.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T00:07:59.251 INFO:teuthology.orchestra.run.smithi079.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T00:07:59.251 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.251 INFO:teuthology.orchestra.run.smithi079.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T00:07:59.251 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T00:07:59.251 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T00:07:59.251 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.251 INFO:teuthology.orchestra.run.smithi079.stdout: # 1m isn't quite enough 2024-09-15T00:07:59.251 INFO:teuthology.orchestra.run.smithi079.stdout: mon_down_mkfs_grace = 2m 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_filestore_osds = false 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout:[client] 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout: rgw cache enabled = true 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable ops log = true 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable usage log = true 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.a] 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.c] 2024-09-15T00:07:59.252 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.b] 2024-09-15T00:07:59.254 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T00:07:59.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T00:07:59.350 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T00:07:59.351 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:59.351 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T00:07:59.367 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:59.367 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T00:07:59.423 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:07:59.423 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:59.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T00:07:59.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T00:07:59.499 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:07:59.499 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:07:59.555 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi079.front.sepia.ceph.com 2024-09-15T00:07:59.555 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:07:59.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T00:07:59.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T00:07:59.591 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:07:59.591 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:07:59.647 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T00:07:59.648 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T00:07:59.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T00:07:59.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T00:07:59.687 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T00:07:59.687 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:07:59.745 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T00:07:59.745 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T00:07:59.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T00:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T00:07:59.810 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T00:07:59.810 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T00:07:59.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T00:07:59.864 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T00:07:59.878 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T00:07:59.878 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-15T00:07:59.878 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T00:07:59.904 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T00:07:59.904 INFO:tasks.ceph:role: osd.0 2024-09-15T00:07:59.905 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:07:59.905 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T00:07:59.977 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:07:59.978 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:07:59.978 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:07:59.978 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:07:59.978 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:07:59.978 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:07:59.978 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:07:59.978 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:07:59.978 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:07:59.978 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:08:00.496 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-15T00:08:00.498 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-15T00:08:00.498 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T00:08:00.652 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T00:08:00.684 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T00:08:00.750 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T00:08:00.751 INFO:tasks.ceph:role: osd.1 2024-09-15T00:08:00.751 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:08:00.751 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T00:08:00.824 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:08:00.825 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:08:00.825 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:08:00.825 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:08:00.825 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:08:00.825 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:08:00.825 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:08:00.825 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:08:00.825 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:08:00.825 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:08:01.273 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-15T00:08:01.276 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-15T00:08:01.276 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T00:08:01.374 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T00:08:01.404 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T00:08:01.470 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T00:08:01.470 INFO:tasks.ceph:role: osd.2 2024-09-15T00:08:01.470 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:08:01.470 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T00:08:01.544 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:08:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:08:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:08:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:08:01.544 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:08:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:08:01.544 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:08:01.544 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:08:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:08:01.545 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:08:02.027 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-15T00:08:02.029 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-15T00:08:02.029 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T00:08:02.168 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T00:08:02.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:08:02.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:02.299+0000 7f24fc411640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T00:08:02.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:02.300+0000 7f24fc411640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T00:08:02.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:02.300+0000 7f24fc411640 -1 bdev(0x55b4c9ccb800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-15T00:08:02.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:02.300+0000 7f24fc411640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T00:08:05.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T00:08:05.984 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:08:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:06.075+0000 7f27fc7db640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T00:08:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:06.075+0000 7f27fc7db640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T00:08:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:06.075+0000 7f27fc7db640 -1 bdev(0x5578a39bd800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-15T00:08:06.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:06.076+0000 7f27fc7db640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T00:08:09.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T00:08:09.274 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:08:09.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:09.364+0000 7f6af17a4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T00:08:09.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:09.365+0000 7f6af17a4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T00:08:09.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:09.365+0000 7f6af17a4640 -1 bdev(0x5561f3e33800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-15T00:08:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:09.365+0000 7f6af17a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T00:08:12.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T00:08:12.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T00:08:12.829 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T00:08:12.829 INFO:tasks.ceph:role: osd.3 2024-09-15T00:08:12.830 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-15T00:08:12.830 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T00:08:12.905 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:08:12.905 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:08:12.905 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:08:12.905 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:08:12.906 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:08:12.906 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:08:12.906 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:08:12.906 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:08:12.906 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:08:12.906 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:08:13.397 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-09-15T00:08:13.399 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-15T00:08:13.399 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-15T00:08:13.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T00:08:13.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T00:08:13.634 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T00:08:13.634 INFO:tasks.ceph:role: osd.4 2024-09-15T00:08:13.635 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-15T00:08:13.635 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:08:13.707 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:08:14.176 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-09-15T00:08:14.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-15T00:08:14.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-15T00:08:14.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T00:08:14.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T00:08:14.366 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T00:08:14.367 INFO:tasks.ceph:role: osd.5 2024-09-15T00:08:14.367 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-15T00:08:14.367 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T00:08:14.439 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T00:08:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T00:08:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T00:08:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T00:08:14.440 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T00:08:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-15T00:08:14.440 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T00:08:14.440 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T00:08:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T00:08:14.440 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T00:08:14.936 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-09-15T00:08:14.938 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-15T00:08:14.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-15T00:08:15.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T00:08:15.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:08:15.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:15.214+0000 7f210260f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T00:08:15.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:15.215+0000 7f210260f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T00:08:15.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:15.215+0000 7f210260f640 -1 bdev(0x55b9dcda5800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-15T00:08:15.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:15.215+0000 7f210260f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T00:08:18.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T00:08:18.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:08:18.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:18.497+0000 7f5f91eb4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T00:08:18.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:18.497+0000 7f5f91eb4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T00:08:18.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:18.497+0000 7f5f91eb4640 -1 bdev(0x55bcde6fd800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-15T00:08:18.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:18.497+0000 7f5f91eb4640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T00:08:21.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T00:08:21.934 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:08:22.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:22.022+0000 7f8e57411640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T00:08:22.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:22.022+0000 7f8e57411640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T00:08:22.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:22.022+0000 7f8e57411640 -1 bdev(0x557492dab800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-15T00:08:22.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-15T00:08:22.022+0000 7f8e57411640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T00:08:25.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T00:08:25.218 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T00:08:25.218 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:08:25.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T00:08:25.249 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:08:25.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T00:08:25.319 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:08:25.319 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T00:08:25.389 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:08:25.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T00:08:25.418 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:08:25.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T00:08:25.487 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:08:25.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T00:08:25.554 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:08:25.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T00:08:25.621 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T00:08:25.621 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T00:08:25.640 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T00:08:25.641 DEBUG:teuthology.orchestra.run.smithi027:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T00:08:25.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T00:08:25.670 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.0] 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBiJeZm4evgERAA0eElQakg2D8BFiRO70eyHg== 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.1] 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBmJeZme+uHBBAAt1STP5ecRIrkXttlZjgPCg== 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.2] 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBpJeZmEpfWFRAAdgbJ7IovoWbVoSpA4FhzhQ== 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.x] 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBfJeZm00UgMBAAzNweQJEiXyQSNdloBnNI3A== 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.3] 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBvJeZmQJvdDBAATCr0opMxpnnVxW33s1v2Qw== 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.4] 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQByJeZm1xnDHRAAqpCjVu6LY9PjbiJunPB3Hw== 2024-09-15T00:08:25.671 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.5] 2024-09-15T00:08:25.672 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB2JeZmXxBjARAAUgXVZhUuE8z40Q+tMl4kZg== 2024-09-15T00:08:25.672 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-15T00:08:25.672 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBfJeZmjtx+MxAAqDQcErPyEjIQaD0C5WAA5w== 2024-09-15T00:08:25.686 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.0] 2024-09-15T00:08:25.686 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBiJeZm4evgERAA0eElQakg2D8BFiRO70eyHg== 2024-09-15T00:08:25.686 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.1] 2024-09-15T00:08:25.686 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBmJeZme+uHBBAAt1STP5ecRIrkXttlZjgPCg== 2024-09-15T00:08:25.686 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.2] 2024-09-15T00:08:25.686 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBpJeZmEpfWFRAAdgbJ7IovoWbVoSpA4FhzhQ== 2024-09-15T00:08:25.686 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.x] 2024-09-15T00:08:25.686 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBfJeZm00UgMBAAzNweQJEiXyQSNdloBnNI3A== 2024-09-15T00:08:25.687 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.3] 2024-09-15T00:08:25.687 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBvJeZmQJvdDBAATCr0opMxpnnVxW33s1v2Qw== 2024-09-15T00:08:25.687 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.4] 2024-09-15T00:08:25.687 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQByJeZm1xnDHRAAqpCjVu6LY9PjbiJunPB3Hw== 2024-09-15T00:08:25.687 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.5] 2024-09-15T00:08:25.687 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQB2JeZmXxBjARAAUgXVZhUuE8z40Q+tMl4kZg== 2024-09-15T00:08:25.687 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2024-09-15T00:08:25.687 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBfJeZmjtx+MxAAqDQcErPyEjIQaD0C5WAA5w== 2024-09-15T00:08:25.688 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:25.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:25.781 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:25.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:25.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:25.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:25.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T00:08:25.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T00:08:25.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:25.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:25.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:25.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:26.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:26.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T00:08:26.101 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T00:08:26.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T00:08:26.157 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T00:08:26.157 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T00:08:26.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T00:08:26.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T00:08:26.306 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T00:08:26.372 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T00:08:26.470 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T00:08:26.496 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T00:08:26.521 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T00:08:26.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T00:08:26.642 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:08:26.645 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T00:08:26.698 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T00:08:26.698 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T00:08:26.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T00:08:26.700 INFO:tasks.ceph.mon.a:Started 2024-09-15T00:08:26.700 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T00:08:26.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T00:08:26.702 INFO:tasks.ceph.mon.c:Started 2024-09-15T00:08:26.702 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T00:08:26.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T00:08:26.740 INFO:tasks.ceph.mon.b:Started 2024-09-15T00:08:26.740 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T00:08:26.740 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T00:08:26.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T00:08:26.742 INFO:tasks.ceph.mgr.x:Started 2024-09-15T00:08:26.743 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T00:08:26.743 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T00:08:32.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:08:32.228+0000 7f62c9c94640 0 --2- 172.21.15.27:0/2083750295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62b00037a0 0x7f62c4131cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T00:08:32.304 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-09-15T00:08:32.317 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T00:08:32.317 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T00:08:32.317 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:08:32.317 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T00:08:32.344 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:08:32.363 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T00:08:32.410 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T00:08:32.411 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T00:08:32.476 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:08:32.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T00:08:32.506 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:08:32.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T00:08:32.575 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-15T00:08:32.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T00:08:32.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 56600dc2-b95d-47a7-b2b1-5ddc54c08341 0 2024-09-15T00:08:32.871 INFO:teuthology.orchestra.run.smithi079.stdout:0 2024-09-15T00:08:32.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new ea8f0155-8472-4e87-89fd-7716d05065d0 1 2024-09-15T00:08:32.913 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:32.911+0000 7f0a92e89040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T00:08:32.955 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:32.954+0000 7f0a92e89040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T00:08:33.079 INFO:teuthology.orchestra.run.smithi079.stdout:1 2024-09-15T00:08:33.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 357d2d59-66f0-47b8-87c8-4acdfe62a07d 2 2024-09-15T00:08:33.113 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:33.111+0000 7f0a92e89040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T00:08:33.166 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:33.165+0000 7f0a92e89040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T00:08:33.321 INFO:teuthology.orchestra.run.smithi079.stdout:2 2024-09-15T00:08:33.333 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 22208f56-d8aa-4a56-a42c-5a990962b860 3 2024-09-15T00:08:33.378 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:33.377+0000 7f0a92e89040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T00:08:33.494 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:33.492+0000 7f0a92e89040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T00:08:33.516 INFO:teuthology.orchestra.run.smithi079.stdout:3 2024-09-15T00:08:33.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new a974629e-9ef0-40a3-94bd-382e0bfa8bb6 4 2024-09-15T00:08:33.706 INFO:teuthology.orchestra.run.smithi079.stdout:4 2024-09-15T00:08:33.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd new 06b73c2d-466a-4c4e-89fa-31d6eeb1af1d 5 2024-09-15T00:08:33.826 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:33.825+0000 7f0a92e89040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T00:08:33.877 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:33.875+0000 7f0a92e89040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T00:08:33.894 INFO:teuthology.orchestra.run.smithi079.stdout:5 2024-09-15T00:08:33.905 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T00:08:33.905 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T00:08:33.908 INFO:tasks.ceph.osd.0:Started 2024-09-15T00:08:33.908 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T00:08:33.908 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T00:08:33.910 INFO:tasks.ceph.osd.1:Started 2024-09-15T00:08:33.910 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T00:08:33.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T00:08:33.913 INFO:tasks.ceph.osd.2:Started 2024-09-15T00:08:33.913 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T00:08:33.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T00:08:33.915 INFO:tasks.ceph.osd.3:Started 2024-09-15T00:08:33.915 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T00:08:33.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T00:08:33.918 INFO:tasks.ceph.osd.4:Started 2024-09-15T00:08:33.918 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T00:08:33.918 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T00:08:33.920 INFO:tasks.ceph.osd.5:Started 2024-09-15T00:08:33.920 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:08:34.038 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:34.036+0000 7f0a92e89040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T00:08:34.107 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:34.107 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"fsid":"4f532d31-28c1-4236-a74e-6392960baf2b","created":"2024-09-15T00:08:32.007191+0000","modified":"2024-09-15T00:08:33.890963+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T00:08:33.890963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"56600dc2-b95d-47a7-b2b1-5ddc54c08341","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ea8f0155-8472-4e87-89fd-7716d05065d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"357d2d59-66f0-47b8-87c8-4acdfe62a07d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"22208f56-d8aa-4a56-a42c-5a990962b860","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a974629e-9ef0-40a3-94bd-382e0bfa8bb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"06b73c2d-466a-4c4e-89fa-31d6eeb1af1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:08:34.118 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T00:08:34.118 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T00:08:34.722 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T00:08:34.885 INFO:teuthology.misc.health.smithi027.stdout: 2024-09-15T00:08:34.885 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":8,"fsid":"4f532d31-28c1-4236-a74e-6392960baf2b","created":"2024-09-15T00:08:32.007191+0000","modified":"2024-09-15T00:08:33.890963+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T00:08:33.890963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"56600dc2-b95d-47a7-b2b1-5ddc54c08341","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ea8f0155-8472-4e87-89fd-7716d05065d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"357d2d59-66f0-47b8-87c8-4acdfe62a07d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"22208f56-d8aa-4a56-a42c-5a990962b860","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a974629e-9ef0-40a3-94bd-382e0bfa8bb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"06b73c2d-466a-4c4e-89fa-31d6eeb1af1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:08:34.895 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-15T00:08:34.967 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:34.965+0000 7f0a92e89040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T00:08:35.019 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.017+0000 7f0a92e89040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T00:08:35.122 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:08:35.120+0000 7f2a429dd640 -1 Falling back to public interface 2024-09-15T00:08:35.135 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:08:35.133+0000 7f96f39c5640 -1 Falling back to public interface 2024-09-15T00:08:35.358 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:08:35.356+0000 7f6fb3c16640 -1 Falling back to public interface 2024-09-15T00:08:35.374 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:08:35.372+0000 7f6e9e01c640 -1 Falling back to public interface 2024-09-15T00:08:35.374 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:08:35.372+0000 7f14a1a1e640 -1 Falling back to public interface 2024-09-15T00:08:35.386 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.384+0000 7f0a92e89040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T00:08:35.472 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.470+0000 7f0a92e89040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T00:08:35.577 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.575+0000 7f0a92e89040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T00:08:35.623 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.621+0000 7f0a92e89040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T00:08:35.642 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:08:35.641+0000 7f57d800d640 -1 Falling back to public interface 2024-09-15T00:08:35.736 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.734+0000 7f0a92e89040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T00:08:35.788 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.786+0000 7f0a92e89040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T00:08:35.831 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.830+0000 7f0a92e89040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T00:08:35.943 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.941+0000 7f0a92e89040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T00:08:35.994 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:35.992+0000 7f0a92e89040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T00:08:36.045 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:36.043+0000 7f0a92e89040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T00:08:36.111 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:36.110+0000 7f0a92e89040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T00:08:36.155 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:36.153+0000 7f0a92e89040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T00:08:36.199 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:36.197+0000 7f0a92e89040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T00:08:36.845 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:36.843+0000 7f0a92e89040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T00:08:37.857 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:37.855+0000 7f0a33754640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T00:08:37.858 INFO:tasks.ceph.mgr.x.smithi079.stderr:2024-09-15T00:08:37.856+0000 7f0a33754640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T00:08:38.905 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:08:38.904+0000 7f2a429dd640 -1 osd.2 0 log_to_monitors true 2024-09-15T00:08:38.926 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:08:38.925+0000 7f14a1a1e640 -1 osd.4 0 log_to_monitors true 2024-09-15T00:08:38.975 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:08:38.974+0000 7f96f39c5640 -1 osd.5 0 log_to_monitors true 2024-09-15T00:08:39.365 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:08:39.363+0000 7f6fb3c16640 -1 osd.1 0 log_to_monitors true 2024-09-15T00:08:39.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:08:39.681+0000 7f57d800d640 -1 osd.0 0 log_to_monitors true 2024-09-15T00:08:39.742 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:08:39.740+0000 7f6e9e01c640 -1 osd.3 0 log_to_monitors true 2024-09-15T00:08:41.501 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T00:08:41.789 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:08:41.787+0000 7f6e999a1640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T00:08:41.800 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:08:41.799+0000 7f6e917c1640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:08:41.806 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:08:41.804+0000 7f96f0168640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T00:08:41.811 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:08:41.808+0000 7f149d45e640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T00:08:41.818 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:08:41.817+0000 7f96e7774640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:08:41.823 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:08:41.822+0000 7f149527e640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:08:41.932 INFO:teuthology.misc.health.smithi027.stdout: 2024-09-15T00:08:41.932 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":10,"fsid":"4f532d31-28c1-4236-a74e-6392960baf2b","created":"2024-09-15T00:08:32.007191+0000","modified":"2024-09-15T00:08:40.856495+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T00:08:33.890963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"56600dc2-b95d-47a7-b2b1-5ddc54c08341","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ea8f0155-8472-4e87-89fd-7716d05065d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"357d2d59-66f0-47b8-87c8-4acdfe62a07d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"22208f56-d8aa-4a56-a42c-5a990962b860","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a974629e-9ef0-40a3-94bd-382e0bfa8bb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"06b73c2d-466a-4c4e-89fa-31d6eeb1af1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:08:41.946 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-15T00:08:41.961 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:08:41.960+0000 7f6fafe67640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T00:08:41.974 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:08:41.973+0000 7f6fa7473640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:08:41.993 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:08:41.992+0000 7f2a3e96c640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T00:08:41.996 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:08:41.995+0000 7f57d425f640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T00:08:42.008 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:08:42.007+0000 7f2a3678c640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:08:42.016 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:08:42.015+0000 7f57cb86b640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:08:48.553 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T00:08:48.845 INFO:teuthology.misc.health.smithi027.stdout: 2024-09-15T00:08:48.846 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":14,"fsid":"4f532d31-28c1-4236-a74e-6392960baf2b","created":"2024-09-15T00:08:32.007191+0000","modified":"2024-09-15T00:08:45.799735+0000","last_up_change":"2024-09-15T00:08:42.794810+0000","last_in_change":"2024-09-15T00:08:33.890963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:08:42.896076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"56600dc2-b95d-47a7-b2b1-5ddc54c08341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6817","nonce":1440638768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6819","nonce":1440638768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6823","nonce":1440638768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6821","nonce":1440638768}]},"public_addr":"172.21.15.27:6817/1440638768","cluster_addr":"172.21.15.27:6819/1440638768","heartbeat_back_addr":"172.21.15.27:6823/1440638768","heartbeat_front_addr":"172.21.15.27:6821/1440638768","state":["exists","up"]},{"osd":1,"uuid":"ea8f0155-8472-4e87-89fd-7716d05065d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6809","nonce":3601510193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6811","nonce":3601510193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6815","nonce":3601510193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6813","nonce":3601510193}]},"public_addr":"172.21.15.27:6809/3601510193","cluster_addr":"172.21.15.27:6811/3601510193","heartbeat_back_addr":"172.21.15.27:6815/3601510193","heartbeat_front_addr":"172.21.15.27:6813/3601510193","state":["exists","up"]},{"osd":2,"uuid":"357d2d59-66f0-47b8-87c8-4acdfe62a07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6801","nonce":3222759256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6803","nonce":3222759256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6807","nonce":3222759256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6805","nonce":3222759256}]},"public_addr":"172.21.15.27:6801/3222759256","cluster_addr":"172.21.15.27:6803/3222759256","heartbeat_back_addr":"172.21.15.27:6807/3222759256","heartbeat_front_addr":"172.21.15.27:6805/3222759256","state":["exists","up"]},{"osd":3,"uuid":"22208f56-d8aa-4a56-a42c-5a990962b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6809","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6811","nonce":675428777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6813","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6816","nonce":675428777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6823","nonce":675428777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6821","nonce":675428777}]},"public_addr":"172.21.15.79:6811/675428777","cluster_addr":"172.21.15.79:6816/675428777","heartbeat_back_addr":"172.21.15.79:6823/675428777","heartbeat_front_addr":"172.21.15.79:6821/675428777","state":["exists","up"]},{"osd":4,"uuid":"a974629e-9ef0-40a3-94bd-382e0bfa8bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6810","nonce":3736884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6814","nonce":3736884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6819","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6820","nonce":3736884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6815","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6817","nonce":3736884138}]},"public_addr":"172.21.15.79:6810/3736884138","cluster_addr":"172.21.15.79:6814/3736884138","heartbeat_back_addr":"172.21.15.79:6820/3736884138","heartbeat_front_addr":"172.21.15.79:6817/3736884138","state":["exists","up"]},{"osd":5,"uuid":"06b73c2d-466a-4c4e-89fa-31d6eeb1af1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6801","nonce":2235671634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6803","nonce":2235671634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6807","nonce":2235671634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6805","nonce":2235671634}]},"public_addr":"172.21.15.79:6801/2235671634","cluster_addr":"172.21.15.79:6803/2235671634","heartbeat_back_addr":"172.21.15.79:6807/2235671634","heartbeat_front_addr":"172.21.15.79:6805/2235671634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.730107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.365297+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.868500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.788781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.892143+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.990037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:08:48.857 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-15T00:08:48.857 INFO:tasks.ceph:Creating RBD pool 2024-09-15T00:08:48.857 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T00:08:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'rbd' created 2024-09-15T00:08:49.842 DEBUG:teuthology.orchestra.run.smithi027:> rbd --cluster ceph pool init rbd 2024-09-15T00:08:52.849 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T00:08:52.881 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T00:08:52.881 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T00:08:52.882 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T00:08:52.882 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T00:08:53.236 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:53.247 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"flags":0,"active_gid":4117,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1584302593},{"type":"v1","addr":"172.21.15.79:6825","nonce":1584302593}]},"active_addr":"172.21.15.79:6825/1584302593","active_change":"2024-09-15T00:08:36.846243+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":1200450359}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3096760296}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":18078947}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":2419462235}]}]} 2024-09-15T00:08:53.249 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T00:08:53.250 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T00:08:53.250 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:08:53.539 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:53.540 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"fsid":"4f532d31-28c1-4236-a74e-6392960baf2b","created":"2024-09-15T00:08:32.007191+0000","modified":"2024-09-15T00:08:52.831466+0000","last_up_change":"2024-09-15T00:08:42.794810+0000","last_in_change":"2024-09-15T00:08:33.890963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:08:42.896076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:08:49.173060+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"56600dc2-b95d-47a7-b2b1-5ddc54c08341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6817","nonce":1440638768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6819","nonce":1440638768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6823","nonce":1440638768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6821","nonce":1440638768}]},"public_addr":"172.21.15.27:6817/1440638768","cluster_addr":"172.21.15.27:6819/1440638768","heartbeat_back_addr":"172.21.15.27:6823/1440638768","heartbeat_front_addr":"172.21.15.27:6821/1440638768","state":["exists","up"]},{"osd":1,"uuid":"ea8f0155-8472-4e87-89fd-7716d05065d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6809","nonce":3601510193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6811","nonce":3601510193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6815","nonce":3601510193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6813","nonce":3601510193}]},"public_addr":"172.21.15.27:6809/3601510193","cluster_addr":"172.21.15.27:6811/3601510193","heartbeat_back_addr":"172.21.15.27:6815/3601510193","heartbeat_front_addr":"172.21.15.27:6813/3601510193","state":["exists","up"]},{"osd":2,"uuid":"357d2d59-66f0-47b8-87c8-4acdfe62a07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6801","nonce":3222759256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6803","nonce":3222759256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6807","nonce":3222759256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6805","nonce":3222759256}]},"public_addr":"172.21.15.27:6801/3222759256","cluster_addr":"172.21.15.27:6803/3222759256","heartbeat_back_addr":"172.21.15.27:6807/3222759256","heartbeat_front_addr":"172.21.15.27:6805/3222759256","state":["exists","up"]},{"osd":3,"uuid":"22208f56-d8aa-4a56-a42c-5a990962b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6809","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6811","nonce":675428777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6813","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6816","nonce":675428777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6823","nonce":675428777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6821","nonce":675428777}]},"public_addr":"172.21.15.79:6811/675428777","cluster_addr":"172.21.15.79:6816/675428777","heartbeat_back_addr":"172.21.15.79:6823/675428777","heartbeat_front_addr":"172.21.15.79:6821/675428777","state":["exists","up"]},{"osd":4,"uuid":"a974629e-9ef0-40a3-94bd-382e0bfa8bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6810","nonce":3736884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6814","nonce":3736884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6819","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6820","nonce":3736884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6815","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6817","nonce":3736884138}]},"public_addr":"172.21.15.79:6810/3736884138","cluster_addr":"172.21.15.79:6814/3736884138","heartbeat_back_addr":"172.21.15.79:6820/3736884138","heartbeat_front_addr":"172.21.15.79:6817/3736884138","state":["exists","up"]},{"osd":5,"uuid":"06b73c2d-466a-4c4e-89fa-31d6eeb1af1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6801","nonce":2235671634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6803","nonce":2235671634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6807","nonce":2235671634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6805","nonce":2235671634}]},"public_addr":"172.21.15.79:6801/2235671634","cluster_addr":"172.21.15.79:6803/2235671634","heartbeat_back_addr":"172.21.15.79:6807/2235671634","heartbeat_front_addr":"172.21.15.79:6805/2235671634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.730107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.365297+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.868500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.788781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.892143+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.990037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:08:53.552 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T00:08:53.552 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:08:53.839 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:53.839 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"fsid":"4f532d31-28c1-4236-a74e-6392960baf2b","created":"2024-09-15T00:08:32.007191+0000","modified":"2024-09-15T00:08:52.831466+0000","last_up_change":"2024-09-15T00:08:42.794810+0000","last_in_change":"2024-09-15T00:08:33.890963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:08:42.896076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:08:49.173060+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"56600dc2-b95d-47a7-b2b1-5ddc54c08341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6817","nonce":1440638768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6819","nonce":1440638768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6823","nonce":1440638768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1440638768},{"type":"v1","addr":"172.21.15.27:6821","nonce":1440638768}]},"public_addr":"172.21.15.27:6817/1440638768","cluster_addr":"172.21.15.27:6819/1440638768","heartbeat_back_addr":"172.21.15.27:6823/1440638768","heartbeat_front_addr":"172.21.15.27:6821/1440638768","state":["exists","up"]},{"osd":1,"uuid":"ea8f0155-8472-4e87-89fd-7716d05065d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6809","nonce":3601510193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6811","nonce":3601510193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6815","nonce":3601510193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6813","nonce":3601510193}]},"public_addr":"172.21.15.27:6809/3601510193","cluster_addr":"172.21.15.27:6811/3601510193","heartbeat_back_addr":"172.21.15.27:6815/3601510193","heartbeat_front_addr":"172.21.15.27:6813/3601510193","state":["exists","up"]},{"osd":2,"uuid":"357d2d59-66f0-47b8-87c8-4acdfe62a07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6801","nonce":3222759256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6803","nonce":3222759256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6807","nonce":3222759256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6805","nonce":3222759256}]},"public_addr":"172.21.15.27:6801/3222759256","cluster_addr":"172.21.15.27:6803/3222759256","heartbeat_back_addr":"172.21.15.27:6807/3222759256","heartbeat_front_addr":"172.21.15.27:6805/3222759256","state":["exists","up"]},{"osd":3,"uuid":"22208f56-d8aa-4a56-a42c-5a990962b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6809","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6811","nonce":675428777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6813","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6816","nonce":675428777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6823","nonce":675428777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6821","nonce":675428777}]},"public_addr":"172.21.15.79:6811/675428777","cluster_addr":"172.21.15.79:6816/675428777","heartbeat_back_addr":"172.21.15.79:6823/675428777","heartbeat_front_addr":"172.21.15.79:6821/675428777","state":["exists","up"]},{"osd":4,"uuid":"a974629e-9ef0-40a3-94bd-382e0bfa8bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6810","nonce":3736884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6814","nonce":3736884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6819","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6820","nonce":3736884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6815","nonce":3736884138},{"type":"v1","addr":"172.21.15.79:6817","nonce":3736884138}]},"public_addr":"172.21.15.79:6810/3736884138","cluster_addr":"172.21.15.79:6814/3736884138","heartbeat_back_addr":"172.21.15.79:6820/3736884138","heartbeat_front_addr":"172.21.15.79:6817/3736884138","state":["exists","up"]},{"osd":5,"uuid":"06b73c2d-466a-4c4e-89fa-31d6eeb1af1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6801","nonce":2235671634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6803","nonce":2235671634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6807","nonce":2235671634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6805","nonce":2235671634}]},"public_addr":"172.21.15.79:6801/2235671634","cluster_addr":"172.21.15.79:6803/2235671634","heartbeat_back_addr":"172.21.15.79:6807/2235671634","heartbeat_front_addr":"172.21.15.79:6805/2235671634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.730107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.365297+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.868500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.788781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.892143+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.990037+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:08:53.853 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T00:08:53.854 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T00:08:53.854 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T00:08:53.854 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T00:08:53.855 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T00:08:53.855 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T00:08:54.012 INFO:teuthology.orchestra.run.smithi027.stdout:47244640262 2024-09-15T00:08:54.012 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T00:08:54.016 INFO:teuthology.orchestra.run.smithi027.stdout:47244640261 2024-09-15T00:08:54.016 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T00:08:54.062 INFO:teuthology.orchestra.run.smithi027.stdout:47244640262 2024-09-15T00:08:54.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T00:08:54.066 INFO:teuthology.orchestra.run.smithi027.stdout:47244640261 2024-09-15T00:08:54.066 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T00:08:54.068 INFO:teuthology.orchestra.run.smithi027.stdout:47244640261 2024-09-15T00:08:54.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T00:08:54.070 INFO:teuthology.orchestra.run.smithi027.stdout:47244640261 2024-09-15T00:08:54.070 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T00:08:54.433 INFO:teuthology.orchestra.run.smithi027.stdout:47244640259 2024-09-15T00:08:54.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.4 2024-09-15T00:08:54.452 INFO:teuthology.orchestra.run.smithi027.stdout:47244640259 2024-09-15T00:08:54.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.1 2024-09-15T00:08:54.507 INFO:teuthology.orchestra.run.smithi027.stdout:47244640260 2024-09-15T00:08:54.519 INFO:teuthology.orchestra.run.smithi027.stdout:47244640259 2024-09-15T00:08:54.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.3 2024-09-15T00:08:54.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.2 2024-09-15T00:08:54.561 INFO:teuthology.orchestra.run.smithi027.stdout:47244640260 2024-09-15T00:08:54.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.0 2024-09-15T00:08:54.592 INFO:teuthology.orchestra.run.smithi027.stdout:47244640259 2024-09-15T00:08:54.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.5 2024-09-15T00:08:55.447 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T00:08:55.466 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T00:08:55.521 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T00:08:55.533 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T00:08:55.574 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T00:08:55.605 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T00:08:55.910 INFO:teuthology.orchestra.run.smithi027.stdout:47244640261 2024-09-15T00:08:55.911 INFO:teuthology.orchestra.run.smithi027.stdout:47244640262 2024-09-15T00:08:55.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.0 2024-09-15T00:08:55.927 DEBUG:teuthology.parallel:result is None 2024-09-15T00:08:55.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.4 2024-09-15T00:08:55.931 DEBUG:teuthology.parallel:result is None 2024-09-15T00:08:55.958 INFO:teuthology.orchestra.run.smithi027.stdout:47244640261 2024-09-15T00:08:55.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.1 2024-09-15T00:08:55.971 DEBUG:teuthology.parallel:result is None 2024-09-15T00:08:55.973 INFO:teuthology.orchestra.run.smithi027.stdout:47244640261 2024-09-15T00:08:55.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.5 2024-09-15T00:08:55.987 DEBUG:teuthology.parallel:result is None 2024-09-15T00:08:56.015 INFO:teuthology.orchestra.run.smithi027.stdout:47244640261 2024-09-15T00:08:56.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2024-09-15T00:08:56.029 DEBUG:teuthology.parallel:result is None 2024-09-15T00:08:56.048 INFO:teuthology.orchestra.run.smithi027.stdout:47244640262 2024-09-15T00:08:56.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.3 2024-09-15T00:08:56.061 DEBUG:teuthology.parallel:result is None 2024-09-15T00:08:56.061 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T00:08:56.061 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:08:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:56.388 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:08:56.400 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T00:08:54.854480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163324,"kb_used_data":1128,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067716,"statfs":{"total":579820584960,"available":579653341184,"internally_reserved":0,"allocated":1155072,"data_stored":1521508,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.033841"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835701+0000","last_change":"2024-09-15T00:08:52.835701+0000","last_active":"2024-09-15T00:08:52.835701+0000","last_peered":"2024-09-15T00:08:52.835701+0000","last_clean":"2024-09-15T00:08:52.835701+0000","last_became_active":"2024-09-15T00:08:50.969842+0000","last_became_peered":"2024-09-15T00:08:50.969842+0000","last_unstale":"2024-09-15T00:08:52.835701+0000","last_undegraded":"2024-09-15T00:08:52.835701+0000","last_fullsized":"2024-09-15T00:08:52.835701+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:00.926947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.842938+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:08:52.842938+0000","last_peered":"2024-09-15T00:08:52.842938+0000","last_clean":"2024-09-15T00:08:52.842938+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:08:52.842938+0000","last_undegraded":"2024-09-15T00:08:52.842938+0000","last_fullsized":"2024-09-15T00:08:52.842938+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835997+0000","last_change":"2024-09-15T00:08:52.835997+0000","last_active":"2024-09-15T00:08:52.835997+0000","last_peered":"2024-09-15T00:08:52.835997+0000","last_clean":"2024-09-15T00:08:52.835997+0000","last_became_active":"2024-09-15T00:08:51.070763+0000","last_became_peered":"2024-09-15T00:08:51.070763+0000","last_unstale":"2024-09-15T00:08:52.835997+0000","last_undegraded":"2024-09-15T00:08:52.835997+0000","last_fullsized":"2024-09-15T00:08:52.835997+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:58.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.842966+0000","last_change":"2024-09-15T00:08:52.842966+0000","last_active":"2024-09-15T00:08:52.842966+0000","last_peered":"2024-09-15T00:08:52.842966+0000","last_clean":"2024-09-15T00:08:52.842966+0000","last_became_active":"2024-09-15T00:08:51.070528+0000","last_became_peered":"2024-09-15T00:08:51.070528+0000","last_unstale":"2024-09-15T00:08:52.842966+0000","last_undegraded":"2024-09-15T00:08:52.842966+0000","last_fullsized":"2024-09-15T00:08:52.842966+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:52:05.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.841091+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:08:52.841091+0000","last_peered":"2024-09-15T00:08:52.841091+0000","last_clean":"2024-09-15T00:08:52.841091+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:08:52.841091+0000","last_undegraded":"2024-09-15T00:08:52.841091+0000","last_fullsized":"2024-09-15T00:08:52.841091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.836839+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:08:52.836839+0000","last_peered":"2024-09-15T00:08:52.836839+0000","last_clean":"2024-09-15T00:08:52.836839+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:08:52.836839+0000","last_undegraded":"2024-09-15T00:08:52.836839+0000","last_fullsized":"2024-09-15T00:08:52.836839+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835672+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:08:52.835672+0000","last_peered":"2024-09-15T00:08:52.835672+0000","last_clean":"2024-09-15T00:08:52.835672+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:08:52.835672+0000","last_undegraded":"2024-09-15T00:08:52.835672+0000","last_fullsized":"2024-09-15T00:08:52.835672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.839513+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:08:52.839513+0000","last_peered":"2024-09-15T00:08:52.839513+0000","last_clean":"2024-09-15T00:08:52.839513+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:08:52.839513+0000","last_undegraded":"2024-09-15T00:08:52.839513+0000","last_fullsized":"2024-09-15T00:08:52.839513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":170,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835116+0000","last_change":"2024-09-15T00:08:44.804415+0000","last_active":"2024-09-15T00:08:52.835116+0000","last_peered":"2024-09-15T00:08:52.835116+0000","last_clean":"2024-09-15T00:08:52.835116+0000","last_became_active":"2024-09-15T00:08:44.804011+0000","last_became_peered":"2024-09-15T00:08:44.804011+0000","last_unstale":"2024-09-15T00:08:52.835116+0000","last_undegraded":"2024-09-15T00:08:52.835116+0000","last_fullsized":"2024-09-15T00:08:52.835116+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:12.388733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:08:56.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:08:56.688 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:56.689 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:08:56.701 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T00:08:54.854480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163324,"kb_used_data":1128,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067716,"statfs":{"total":579820584960,"available":579653341184,"internally_reserved":0,"allocated":1155072,"data_stored":1521508,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.033841"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835701+0000","last_change":"2024-09-15T00:08:52.835701+0000","last_active":"2024-09-15T00:08:52.835701+0000","last_peered":"2024-09-15T00:08:52.835701+0000","last_clean":"2024-09-15T00:08:52.835701+0000","last_became_active":"2024-09-15T00:08:50.969842+0000","last_became_peered":"2024-09-15T00:08:50.969842+0000","last_unstale":"2024-09-15T00:08:52.835701+0000","last_undegraded":"2024-09-15T00:08:52.835701+0000","last_fullsized":"2024-09-15T00:08:52.835701+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:00.926947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.842938+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:08:52.842938+0000","last_peered":"2024-09-15T00:08:52.842938+0000","last_clean":"2024-09-15T00:08:52.842938+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:08:52.842938+0000","last_undegraded":"2024-09-15T00:08:52.842938+0000","last_fullsized":"2024-09-15T00:08:52.842938+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835997+0000","last_change":"2024-09-15T00:08:52.835997+0000","last_active":"2024-09-15T00:08:52.835997+0000","last_peered":"2024-09-15T00:08:52.835997+0000","last_clean":"2024-09-15T00:08:52.835997+0000","last_became_active":"2024-09-15T00:08:51.070763+0000","last_became_peered":"2024-09-15T00:08:51.070763+0000","last_unstale":"2024-09-15T00:08:52.835997+0000","last_undegraded":"2024-09-15T00:08:52.835997+0000","last_fullsized":"2024-09-15T00:08:52.835997+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:58.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.842966+0000","last_change":"2024-09-15T00:08:52.842966+0000","last_active":"2024-09-15T00:08:52.842966+0000","last_peered":"2024-09-15T00:08:52.842966+0000","last_clean":"2024-09-15T00:08:52.842966+0000","last_became_active":"2024-09-15T00:08:51.070528+0000","last_became_peered":"2024-09-15T00:08:51.070528+0000","last_unstale":"2024-09-15T00:08:52.842966+0000","last_undegraded":"2024-09-15T00:08:52.842966+0000","last_fullsized":"2024-09-15T00:08:52.842966+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:52:05.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.841091+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:08:52.841091+0000","last_peered":"2024-09-15T00:08:52.841091+0000","last_clean":"2024-09-15T00:08:52.841091+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:08:52.841091+0000","last_undegraded":"2024-09-15T00:08:52.841091+0000","last_fullsized":"2024-09-15T00:08:52.841091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.836839+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:08:52.836839+0000","last_peered":"2024-09-15T00:08:52.836839+0000","last_clean":"2024-09-15T00:08:52.836839+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:08:52.836839+0000","last_undegraded":"2024-09-15T00:08:52.836839+0000","last_fullsized":"2024-09-15T00:08:52.836839+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835672+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:08:52.835672+0000","last_peered":"2024-09-15T00:08:52.835672+0000","last_clean":"2024-09-15T00:08:52.835672+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:08:52.835672+0000","last_undegraded":"2024-09-15T00:08:52.835672+0000","last_fullsized":"2024-09-15T00:08:52.835672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.839513+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:08:52.839513+0000","last_peered":"2024-09-15T00:08:52.839513+0000","last_clean":"2024-09-15T00:08:52.839513+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:08:52.839513+0000","last_undegraded":"2024-09-15T00:08:52.839513+0000","last_fullsized":"2024-09-15T00:08:52.839513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":170,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835116+0000","last_change":"2024-09-15T00:08:44.804415+0000","last_active":"2024-09-15T00:08:52.835116+0000","last_peered":"2024-09-15T00:08:52.835116+0000","last_clean":"2024-09-15T00:08:52.835116+0000","last_became_active":"2024-09-15T00:08:44.804011+0000","last_became_peered":"2024-09-15T00:08:44.804011+0000","last_unstale":"2024-09-15T00:08:52.835116+0000","last_undegraded":"2024-09-15T00:08:52.835116+0000","last_fullsized":"2024-09-15T00:08:52.835116+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:12.388733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:08:56.702 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T00:08:56.702 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T00:08:56.702 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T00:08:56.702 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T00:08:57.030 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:57.030 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T00:08:57.043 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T00:08:57.043 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-15T00:08:57.061 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T00:08:57.062 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-15T00:08:57.062 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-15T00:08:57.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-15T00:08:57.371 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-15T00:08:57.371 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T00:08:57.371 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:08:57.656 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:08:57.669 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T00:08:56.854984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163324,"kb_used_data":1128,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067716,"statfs":{"total":579820584960,"available":579653341184,"internally_reserved":0,"allocated":1155072,"data_stored":1521508,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001439"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835701+0000","last_change":"2024-09-15T00:08:52.835701+0000","last_active":"2024-09-15T00:08:52.835701+0000","last_peered":"2024-09-15T00:08:52.835701+0000","last_clean":"2024-09-15T00:08:52.835701+0000","last_became_active":"2024-09-15T00:08:50.969842+0000","last_became_peered":"2024-09-15T00:08:50.969842+0000","last_unstale":"2024-09-15T00:08:52.835701+0000","last_undegraded":"2024-09-15T00:08:52.835701+0000","last_fullsized":"2024-09-15T00:08:52.835701+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:00.926947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.842938+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:08:52.842938+0000","last_peered":"2024-09-15T00:08:52.842938+0000","last_clean":"2024-09-15T00:08:52.842938+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:08:52.842938+0000","last_undegraded":"2024-09-15T00:08:52.842938+0000","last_fullsized":"2024-09-15T00:08:52.842938+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835997+0000","last_change":"2024-09-15T00:08:52.835997+0000","last_active":"2024-09-15T00:08:52.835997+0000","last_peered":"2024-09-15T00:08:52.835997+0000","last_clean":"2024-09-15T00:08:52.835997+0000","last_became_active":"2024-09-15T00:08:51.070763+0000","last_became_peered":"2024-09-15T00:08:51.070763+0000","last_unstale":"2024-09-15T00:08:52.835997+0000","last_undegraded":"2024-09-15T00:08:52.835997+0000","last_fullsized":"2024-09-15T00:08:52.835997+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:58.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.842966+0000","last_change":"2024-09-15T00:08:52.842966+0000","last_active":"2024-09-15T00:08:52.842966+0000","last_peered":"2024-09-15T00:08:52.842966+0000","last_clean":"2024-09-15T00:08:52.842966+0000","last_became_active":"2024-09-15T00:08:51.070528+0000","last_became_peered":"2024-09-15T00:08:51.070528+0000","last_unstale":"2024-09-15T00:08:52.842966+0000","last_undegraded":"2024-09-15T00:08:52.842966+0000","last_fullsized":"2024-09-15T00:08:52.842966+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:52:05.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.841091+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:08:52.841091+0000","last_peered":"2024-09-15T00:08:52.841091+0000","last_clean":"2024-09-15T00:08:52.841091+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:08:52.841091+0000","last_undegraded":"2024-09-15T00:08:52.841091+0000","last_fullsized":"2024-09-15T00:08:52.841091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.836839+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:08:52.836839+0000","last_peered":"2024-09-15T00:08:52.836839+0000","last_clean":"2024-09-15T00:08:52.836839+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:08:52.836839+0000","last_undegraded":"2024-09-15T00:08:52.836839+0000","last_fullsized":"2024-09-15T00:08:52.836839+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835672+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:08:52.835672+0000","last_peered":"2024-09-15T00:08:52.835672+0000","last_clean":"2024-09-15T00:08:52.835672+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:08:52.835672+0000","last_undegraded":"2024-09-15T00:08:52.835672+0000","last_fullsized":"2024-09-15T00:08:52.835672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.839513+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:08:52.839513+0000","last_peered":"2024-09-15T00:08:52.839513+0000","last_clean":"2024-09-15T00:08:52.839513+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:08:52.839513+0000","last_undegraded":"2024-09-15T00:08:52.839513+0000","last_fullsized":"2024-09-15T00:08:52.839513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":170,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835116+0000","last_change":"2024-09-15T00:08:44.804415+0000","last_active":"2024-09-15T00:08:52.835116+0000","last_peered":"2024-09-15T00:08:52.835116+0000","last_clean":"2024-09-15T00:08:52.835116+0000","last_became_active":"2024-09-15T00:08:44.804011+0000","last_became_peered":"2024-09-15T00:08:44.804011+0000","last_unstale":"2024-09-15T00:08:52.835116+0000","last_undegraded":"2024-09-15T00:08:52.835116+0000","last_fullsized":"2024-09-15T00:08:52.835116+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:12.388733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:08:57.670 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:08:57.958 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:57.959 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:08:57.971 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T00:08:56.854984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163324,"kb_used_data":1128,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067716,"statfs":{"total":579820584960,"available":579653341184,"internally_reserved":0,"allocated":1155072,"data_stored":1521508,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001439"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835701+0000","last_change":"2024-09-15T00:08:52.835701+0000","last_active":"2024-09-15T00:08:52.835701+0000","last_peered":"2024-09-15T00:08:52.835701+0000","last_clean":"2024-09-15T00:08:52.835701+0000","last_became_active":"2024-09-15T00:08:50.969842+0000","last_became_peered":"2024-09-15T00:08:50.969842+0000","last_unstale":"2024-09-15T00:08:52.835701+0000","last_undegraded":"2024-09-15T00:08:52.835701+0000","last_fullsized":"2024-09-15T00:08:52.835701+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:00.926947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.842938+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:08:52.842938+0000","last_peered":"2024-09-15T00:08:52.842938+0000","last_clean":"2024-09-15T00:08:52.842938+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:08:52.842938+0000","last_undegraded":"2024-09-15T00:08:52.842938+0000","last_fullsized":"2024-09-15T00:08:52.842938+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835997+0000","last_change":"2024-09-15T00:08:52.835997+0000","last_active":"2024-09-15T00:08:52.835997+0000","last_peered":"2024-09-15T00:08:52.835997+0000","last_clean":"2024-09-15T00:08:52.835997+0000","last_became_active":"2024-09-15T00:08:51.070763+0000","last_became_peered":"2024-09-15T00:08:51.070763+0000","last_unstale":"2024-09-15T00:08:52.835997+0000","last_undegraded":"2024-09-15T00:08:52.835997+0000","last_fullsized":"2024-09-15T00:08:52.835997+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:58.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.842966+0000","last_change":"2024-09-15T00:08:52.842966+0000","last_active":"2024-09-15T00:08:52.842966+0000","last_peered":"2024-09-15T00:08:52.842966+0000","last_clean":"2024-09-15T00:08:52.842966+0000","last_became_active":"2024-09-15T00:08:51.070528+0000","last_became_peered":"2024-09-15T00:08:51.070528+0000","last_unstale":"2024-09-15T00:08:52.842966+0000","last_undegraded":"2024-09-15T00:08:52.842966+0000","last_fullsized":"2024-09-15T00:08:52.842966+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:52:05.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.841091+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:08:52.841091+0000","last_peered":"2024-09-15T00:08:52.841091+0000","last_clean":"2024-09-15T00:08:52.841091+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:08:52.841091+0000","last_undegraded":"2024-09-15T00:08:52.841091+0000","last_fullsized":"2024-09-15T00:08:52.841091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.836839+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:08:52.836839+0000","last_peered":"2024-09-15T00:08:52.836839+0000","last_clean":"2024-09-15T00:08:52.836839+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:08:52.836839+0000","last_undegraded":"2024-09-15T00:08:52.836839+0000","last_fullsized":"2024-09-15T00:08:52.836839+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835672+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:08:52.835672+0000","last_peered":"2024-09-15T00:08:52.835672+0000","last_clean":"2024-09-15T00:08:52.835672+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:08:52.835672+0000","last_undegraded":"2024-09-15T00:08:52.835672+0000","last_fullsized":"2024-09-15T00:08:52.835672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.839513+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:08:52.839513+0000","last_peered":"2024-09-15T00:08:52.839513+0000","last_clean":"2024-09-15T00:08:52.839513+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:08:52.839513+0000","last_undegraded":"2024-09-15T00:08:52.839513+0000","last_fullsized":"2024-09-15T00:08:52.839513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":170,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T00:08:52.835116+0000","last_change":"2024-09-15T00:08:44.804415+0000","last_active":"2024-09-15T00:08:52.835116+0000","last_peered":"2024-09-15T00:08:52.835116+0000","last_clean":"2024-09-15T00:08:52.835116+0000","last_became_active":"2024-09-15T00:08:44.804011+0000","last_became_peered":"2024-09-15T00:08:44.804011+0000","last_unstale":"2024-09-15T00:08:52.835116+0000","last_undegraded":"2024-09-15T00:08:52.835116+0000","last_fullsized":"2024-09-15T00:08:52.835116+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:12.388733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640262,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:08:57.971 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T00:08:57.972 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T00:08:58.260 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 19 2024-09-15T00:08:58.260 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 4f532d31-28c1-4236-a74e-6392960baf2b 2024-09-15T00:08:58.260 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-15T00:08:32.007191+0000 2024-09-15T00:08:58.260 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-09-15T00:08:56.785774+0000 2024-09-15T00:08:58.260 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T00:08:58.260 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 4 2024-09-15T00:08:58.260 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client jewel 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release squid 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 14 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 6 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6816/1440638768,v1:172.21.15.27:6817/1440638768] [v2:172.21.15.27:6818/1440638768,v1:172.21.15.27:6819/1440638768] exists,up 56600dc2-b95d-47a7-b2b1-5ddc54c08341 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6808/3601510193,v1:172.21.15.27:6809/3601510193] [v2:172.21.15.27:6810/3601510193,v1:172.21.15.27:6811/3601510193] exists,up ea8f0155-8472-4e87-89fd-7716d05065d0 2024-09-15T00:08:58.261 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6800/3222759256,v1:172.21.15.27:6801/3222759256] [v2:172.21.15.27:6802/3222759256,v1:172.21.15.27:6803/3222759256] exists,up 357d2d59-66f0-47b8-87c8-4acdfe62a07d 2024-09-15T00:08:58.262 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6809/675428777,v1:172.21.15.79:6811/675428777] [v2:172.21.15.79:6813/675428777,v1:172.21.15.79:6816/675428777] exists,up 22208f56-d8aa-4a56-a42c-5a990962b860 2024-09-15T00:08:58.262 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/3736884138,v1:172.21.15.79:6810/3736884138] [v2:172.21.15.79:6812/3736884138,v1:172.21.15.79:6814/3736884138] exists,up a974629e-9ef0-40a3-94bd-382e0bfa8bb6 2024-09-15T00:08:58.262 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/2235671634,v1:172.21.15.79:6801/2235671634] [v2:172.21.15.79:6802/2235671634,v1:172.21.15.79:6803/2235671634] exists,up 06b73c2d-466a-4c4e-89fa-31d6eeb1af1d 2024-09-15T00:08:58.272 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6816/1440638768,v1:172.21.15.27:6817/1440638768] [v2:172.21.15.27:6818/1440638768,v1:172.21.15.27:6819/1440638768] exists,up 56600dc2-b95d-47a7-b2b1-5ddc54c08341', 'osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6808/3601510193,v1:172.21.15.27:6809/3601510193] [v2:172.21.15.27:6810/3601510193,v1:172.21.15.27:6811/3601510193] exists,up ea8f0155-8472-4e87-89fd-7716d05065d0', 'osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6800/3222759256,v1:172.21.15.27:6801/3222759256] [v2:172.21.15.27:6802/3222759256,v1:172.21.15.27:6803/3222759256] exists,up 357d2d59-66f0-47b8-87c8-4acdfe62a07d', 'osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6809/675428777,v1:172.21.15.79:6811/675428777] [v2:172.21.15.79:6813/675428777,v1:172.21.15.79:6816/675428777] exists,up 22208f56-d8aa-4a56-a42c-5a990962b860', 'osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/3736884138,v1:172.21.15.79:6810/3736884138] [v2:172.21.15.79:6812/3736884138,v1:172.21.15.79:6814/3736884138] exists,up a974629e-9ef0-40a3-94bd-382e0bfa8bb6', 'osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/2235671634,v1:172.21.15.79:6801/2235671634] [v2:172.21.15.79:6802/2235671634,v1:172.21.15.79:6803/2235671634] exists,up 06b73c2d-466a-4c4e-89fa-31d6eeb1af1d'] 2024-09-15T00:08:58.273 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-15T00:08:58.375 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:08:58.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mon.a", 2024-09-15T00:08:58.375 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster": "ceph", 2024-09-15T00:08:58.375 INFO:teuthology.orchestra.run.smithi027.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "admin_socket_mode": "", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_client_required": "cephx, none", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_cluster_required": "cephx", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_debug": "true", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_service_required": "cephx", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_supported": "cephx", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_aio": "true", 2024-09-15T00:08:58.376 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_aio_poll_ms": "250", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_aio_reap_max": "16", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_async_discard": "false", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_block_size": "4096", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_debug_aio": "false", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_debug_inflight_ios": "false", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_enable_discard": "false", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_flock_retry": "3", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_inject_crash": "0", 2024-09-15T00:08:58.377 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_ioring": "false", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_ioring_hipri": "false", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_type": "", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_alloc_size": "1048576", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_allocator": "hybrid", 2024-09-15T00:08:58.378 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_buffered_io": "true", 2024-09-15T00:08:58.379 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_check_for_zeros": "false", 2024-09-15T00:08:58.379 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-15T00:08:58.379 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-15T00:08:58.379 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_compact_log_sync": "false", 2024-09-15T00:08:58.379 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_max_log_runway": "4194304", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_max_prefetch": "1048576", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_min_flush_size": "524288", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_min_log_runway": "1048576", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_replay_recovery": "false", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-15T00:08:58.380 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_sync_write": "false", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_allocation_from_file": "true", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_allocator": "hybrid", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bdev_label_multi": "true", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-15T00:08:58.381 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_create": "true", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_db_create": "false", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_db_path": "", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_db_size": "0", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_path": "", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_preallocate_file": "false", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_size": "107374182400", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_wal_create": "false", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_wal_path": "", 2024-09-15T00:08:58.382 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_wal_size": "100663296", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bluefs": "true", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_autotune": "true", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-15T00:08:58.383 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_size": "0", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_type": "2q", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_clone_cow": "true", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-15T00:08:58.384 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_mode": "none", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_csum_type": "crc32c", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_freelist": "false", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-15T00:08:58.385 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_small_allocations": "0", 2024-09-15T00:08:58.386 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_default_buffered_read": "true", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_default_buffered_write": "false", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fail_eio": "false", 2024-09-15T00:08:58.387 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_mount": "false", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_umount": "false", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-15T00:08:58.388 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_ignore_data_csum": "false", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_log_op_age": "5.000000", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_alloc_size": "0", 2024-09-15T00:08:58.389 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_blob_size": "0", 2024-09-15T00:08:58.390 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-15T00:08:58.390 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-15T00:08:58.390 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-15T00:08:58.390 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_deferred_txc": "32", 2024-09-15T00:08:58.390 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_min_alloc_size": "0", 2024-09-15T00:08:58.390 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-15T00:08:58.390 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-15T00:08:58.390 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_nid_prealloc": "1024", 2024-09-15T00:08:58.390 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_retry_disk_reads": "3", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_rocksdb_cf": "true", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-15T00:08:58.391 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_spdk_mem": "512", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_tracing": "false", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-15T00:08:58.392 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_zero_block_detection": "false", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-15T00:08:58.393 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-15T00:08:58.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-15T00:08:58.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-15T00:08:58.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-15T00:08:58.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-15T00:08:58.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-15T00:08:58.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-15T00:08:58.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-15T00:08:58.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-15T00:08:58.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_cluster_require_signatures": "false", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_cluster_require_version": "2", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_require_signatures": "false", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_require_version": "2", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_service_require_signatures": "false", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_service_require_version": "2", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_sign_messages": "true", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "chdir": "", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client_acl_type": "", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client_asio_thread_count": "2", 2024-09-15T00:08:58.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client_cache_mid": "0.750000", 2024-09-15T00:08:58.396 INFO:teuthology.orchestra.run.smithi027.stdout: "client_cache_size": "16384", 2024-09-15T00:08:58.396 INFO:teuthology.orchestra.run.smithi027.stdout: "client_caps_release_delay": "5", 2024-09-15T00:08:58.396 INFO:teuthology.orchestra.run.smithi027.stdout: "client_check_pool_perm": "true", 2024-09-15T00:08:58.396 INFO:teuthology.orchestra.run.smithi027.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-15T00:08:58.396 INFO:teuthology.orchestra.run.smithi027.stdout: "client_debug_force_sync_read": "false", 2024-09-15T00:08:58.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_debug_getattr_caps": "false", 2024-09-15T00:08:58.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_debug_inject_features": "", 2024-09-15T00:08:58.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_debug_inject_tick_delay": "0", 2024-09-15T00:08:58.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-15T00:08:58.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_die_on_failed_remount": "false", 2024-09-15T00:08:58.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_dirsize_rbytes": "true", 2024-09-15T00:08:58.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_force_lazyio": "false", 2024-09-15T00:08:58.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_fs": "", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_inject_release_failure": "false", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_max_inline_size": "4096", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mds_namespace": "", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_metadata": "", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mount_gid": "-1", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mount_timeout": "300", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mount_uid": "-1", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mountpoint": "/", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_notify_timeout": "10", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc": "true", 2024-09-15T00:08:58.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_max_dirty": "104857600", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_max_objects": "1000", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_size": "209715200", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_target_dirty": "8388608", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_permissions": "true", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_quota": "true", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_quota_df": "true", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_readahead_max_bytes": "0", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_readahead_max_periods": "4", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_readahead_min": "131072", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_reconnect_stale": "false", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_shutdown_timeout": "30", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_snapdir": ".snap", 2024-09-15T00:08:58.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_tick_interval": "1", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_trace": "", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_try_dentry_invalidate": "false", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_use_faked_inos": "false", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_use_random_mds": "false", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_graylog": "false", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_graylog_port": "12201", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_monitors": "default=true", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_syslog": "false", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_syslog_level": "info", 2024-09-15T00:08:58.400 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "-", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_network": "", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_network_interface": "", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "compressor_zlib_isal": "false", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "compressor_zlib_level": "5", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "compressor_zlib_winsize": "-15", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "compressor_zstd_level": "1", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_alien_op_num_threads": "6", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-15T00:08:58.401 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_osd_stat_interval": "0", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_seastar_cpu_cores": "", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_seastar_num_threads": "0", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_location": "", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_location_hook": "", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_location_hook_timeout": "10", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "daemonize": "false", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "daos_pool": "tank", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-15T00:08:58.402 INFO:teuthology.orchestra.run.smithi027.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_alienstore": "0/5", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_allow_any_pool_priority": "false", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_asok": "1/5", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_asok_assert_abort": "false", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_asserts_on_shutdown": "true", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_auth": "20/20", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_bdev": "1/3", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_bluefs": "1/5", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_bluestore": "1/5", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_buffer": "0/1", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_ceph_exporter": "1/5", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_cephfs_mirror": "0/5", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_cephsqlite": "0/5", 2024-09-15T00:08:58.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_client": "0/5", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_compressor": "1/5", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_context": "0/1", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_crimson_interrupt": "0/5", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_crush": "1/1", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_crypto": "1/5", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_cyanstore": "0/5", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_deliberately_leak_memory": "false", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_disable_randomized_ping": "false", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_dpdk": "1/5", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_eventtrace": "1/5", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_filer": "0/1", 2024-09-15T00:08:58.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_filestore": "1/3", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_finisher": "1/1", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_fuse": "1/5", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_heartbeat_testing_span": "0", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_heartbeatmap": "1/5", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_javaclient": "1/5", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_journal": "1/3", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_journaler": "0/5", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_kstore": "1/5", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_lockdep": "0/1", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mclock": "1/5", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds": "1/5", 2024-09-15T00:08:58.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_balancer": "1/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_locker": "1/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_log": "1/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_log_expire": "1/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_migrator": "1/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_quiesce": "3/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_memstore": "1/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mgr": "2/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mgrc": "1/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mon": "20/20", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_monc": "0/10", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_ms": "1/1", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_none": "0/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_objclass": "0/5", 2024-09-15T00:08:58.406 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_objectcacher": "0/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_objecter": "0/1", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_optracker": "0/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_osd": "1/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_paxos": "20/20", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_perfcounter": "1/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_prioritycache": "1/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rados": "0/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rbd": "0/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rbd_mirror": "0/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rbd_pwl": "0/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rbd_replay": "0/5", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_refs": "0/0", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_reserver": "1/1", 2024-09-15T00:08:58.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw": "1/5", 2024-09-15T00:08:58.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_access": "1/5", 2024-09-15T00:08:58.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_datacache": "1/5", 2024-09-15T00:08:58.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_dbstore": "1/5", 2024-09-15T00:08:58.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_flight": "1/5", 2024-09-15T00:08:58.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-15T00:08:58.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_sync": "1/5", 2024-09-15T00:08:58.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rocksdb": "4/5", 2024-09-15T00:08:58.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore": "0/5", 2024-09-15T00:08:58.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_backref": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_cache": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_cleaner": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_device": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_epm": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_journal": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_lba": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_odata": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_omap": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_onode": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_t": "0/5", 2024-09-15T00:08:58.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_tm": "0/5", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_striper": "0/1", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_test": "0/5", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_throttle": "1/1", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_timer": "0/1", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_tp": "0/5", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_trace": "1/5", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "device_failure_prediction_mode": "none", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "err_to_graylog": "false", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "err_to_journald": "false", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "err_to_stderr": "true", 2024-09-15T00:08:58.410 INFO:teuthology.orchestra.run.smithi027.stdout: "err_to_syslog": "false", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event_tracing": "false", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_addr": "0.0.0.0", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_http_port": "9926", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_prio_limit": "5", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_sort_metrics": "true", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_stats_period": "5", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "fake_statfs_for_testing": "0", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "fatal_signal_handlers": "true", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "filer_max_purge_ops": "10", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "filer_max_truncate_ops": "128", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_apply_finisher_threads": "1", 2024-09-15T00:08:58.411 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_blackhole": "false", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_btrfs_clone_range": "true", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_btrfs_snap": "true", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_caller_concurrency": "10", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_collect_device_partition_information": "true", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_commit_timeout": "600.000000", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_debug_inject_read_err": "false", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_debug_omap_check": "false", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_debug_verify_split": "false", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_dump_file": "", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-15T00:08:58.412 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fadvise": "true", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fail_eio": "true", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fd_cache_shards": "16", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fd_cache_size": "128", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fiemap": "false", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fiemap_threshold": "4096", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_inject_stall": "0", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_journal_parallel": "false", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_journal_trailing": "false", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_journal_writeahead": "false", 2024-09-15T00:08:58.413 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_kill_at": "0", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattrs": "0", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_xattr_value_size": "0", 2024-09-15T00:08:58.414 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_merge_threshold": "-10", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_odsync_write": "false", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_omap_backend": "rocksdb", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_omap_backend_path": "", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_op_thread_timeout": "60", 2024-09-15T00:08:58.415 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_op_threads": "2", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_punch_hole": "false", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_ops": "50", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-15T00:08:58.416 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_seek_data_hole": "false", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_sloppy_crc": "false", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_splice": "false", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_split_multiple": "2", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_split_rand_factor": "20", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_update_to": "1000", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-15T00:08:58.417 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_enable": "true", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_xfs_extsize": "false", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_zfs_snap": "false", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "fio_dir": "/tmp/fio", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "4f532d31-28c1-4236-a74e-6392960baf2b", 2024-09-15T00:08:58.418 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_allow_other": "true", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_atomic_o_trunc": "true", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_big_writes": "true", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_debug": "false", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_default_permissions": "false", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_disable_pagecache": "false", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_max_write": "0", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_multithreaded": "true", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_require_active_mds": "true", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_set_user_groups": "true", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_splice_move": "true", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_splice_read": "true", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_splice_write": "true", 2024-09-15T00:08:58.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_use_invalidate_cb": "true", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "gss_target_name": "ceph", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_file": "", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_inject_failure": "0", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_interval": "5", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "host": "smithi027", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-15T00:08:58.420 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "inject_early_sigterm": "false", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "jaeger_agent_port": "6799", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "jaeger_tracing_enable": "false", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_aio": "true", 2024-09-15T00:08:58.421 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_align_min_size": "65536", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_block_align": "true", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_block_size": "4096", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_dio": "true", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_discard": "false", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_force_aio": "false", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_ignore_corruption": "false", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_max_write_bytes": "10485760", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_max_write_entries": "100", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_replay_from": "0", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-15T00:08:58.422 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_write_header_frequency": "0", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_zero_on_create": "false", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "journaler_prefetch_periods": "10", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "journaler_prezero_periods": "5", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "journaler_write_head_interval": "15", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "keyfile": "", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_backend": "rocksdb", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_default_stripe_size": "65536", 2024-09-15T00:08:58.423 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_fsck_on_mount": "false", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_max_bytes": "67108864", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_max_ops": "512", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_nid_prealloc": "1024", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_onode_map_size": "1024", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_sync_submit_transaction": "false", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_sync_transaction": "false", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "librados_thread_count": "2", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "lockdep": "false", 2024-09-15T00:08:58.424 INFO:teuthology.orchestra.run.smithi027.stdout: "lockdep_force_backtrace": "false", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_coarse_timestamps": "true", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_flush_on_exit": "false", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_graylog_host": "127.0.0.1", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_graylog_port": "12201", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_max_new": "1000", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_max_recent": "10000", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_stderr_prefix": "", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_stop_at_utilization": "0.970000", 2024-09-15T00:08:58.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": "true", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_graylog": "false", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_journald": "false", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_stderr": "false", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_syslog": "false", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "max_rotating_auth_attempts": "10", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_action_on_write_error": "1", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_alternate_name_max": "8192", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_asio_thread_count": "2", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_export_pin": "true", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_fragment_dirs": "true", 2024-09-15T00:08:58.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_fragment_interval": "5", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_interval": "10", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_max": "-1", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_max_until": "-1", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_merge_size": "50", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_midchunk": "0.300000", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_min_start": "0.200000", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_minchunk": "0.001000", 2024-09-15T00:08:58.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_mode": "0", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_need_max": "1.200000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_need_min": "0.800000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_overload_epochs": "2", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_split_bits": "3", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_split_size": "10000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_target_decay": "10.000000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_beacon_grace": "15.000000", 2024-09-15T00:08:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_beacon_interval": "4.000000", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_mid": "0.700000", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_delay": "0", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_release_free_interval": "10", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_reservation": "0.050000", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_trim_interval": "1", 2024-09-15T00:08:58.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_trim_threshold": "262144", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_client_prealloc_inos": "1000", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_connect_bootstrapping": "false", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_damage_table_max_entries": "10000", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_debug_auth_pins": "false", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_debug_frag": "false", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_debug_scatterstat": "false", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_debug_subtrees": "false", 2024-09-15T00:08:58.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_decay_halflife": "5.000000", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_default_dir_hash": "2", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_defer_session_stale": "true", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_deny_all_reconnect": "false", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dir_keys_per_op": "16384", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dir_max_commit_size": "10", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dir_max_entries": "0", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dir_prefetch": "true", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dump_cache_on_map": "false", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-15T00:08:58.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_early_reply": "true", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_enable_op_tracker": "true", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_enforce_unique_name": "true", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_export_ephemeral_random": "true", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-15T00:08:58.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_health_summarize_threshold": "10", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_health_dummy": "false", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_migrator_session_race": "false", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_join_fs": "", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_journal_format": "1", 2024-09-15T00:08:58.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_create_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_export_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_import_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_journal_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_journal_expire_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_journal_replay_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_link_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_mdstable_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_openc_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_rename_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_shutdown_at": "0", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_event_large_threshold": "524288", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_events_per_segment": "1024", 2024-09-15T00:08:58.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_max_events": "-1", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_max_segments": "128", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_pause": "false", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_segment_size": "0", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_trim_threshold": "128", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_warn_factor": "2.000000", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_caps_per_client": "1048576", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_completed_flushes": "100000", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_completed_requests": "100000", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_export_size": "20971520", 2024-09-15T00:08:58.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_file_recover": "32", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_purge_files": "64", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_purge_ops": "8192", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_snaps_per_dir": "100", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_metrics_update_interval": "2", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_min_caps_per_client": "100", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_min_caps_working_set": "10000", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_numa_node": "-1", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_complaint_time": "30.000000", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_history_duration": "600", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_history_size": "20", 2024-09-15T00:08:58.436 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_history_slow_op_size": "20", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_log_threshold": "5", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_ping_grace": "15", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_ping_interval": "5", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_max_caps": "30000", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_warning_threshold": "262144", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-15T00:08:58.437 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_replay_interval": "1.000000", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_root_ino_gid": "0", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_root_ino_uid": "0", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-15T00:08:58.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_shutdown_check": "0", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_skip_ino": "0", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_snap_max_uid": "4294967294", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_snap_min_uid": "0", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_snap_rstat": "false", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_standby_replay_damaged": "false", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_symlink_recovery": "true", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_thrash_exports": "0", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_thrash_fragments": "0", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_tick_interval": "5.000000", 2024-09-15T00:08:58.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_valgrind_exit": "false", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_verify_backtrace": "1", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_verify_scatter": "false", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_wipe_sessions": "false", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool_debug": "false", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "memstore_device_bytes": "1073741824", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "memstore_page_set": "false", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "memstore_page_size": "65536", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_client_bytes": "134217728", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_client_messages": "512", 2024-09-15T00:08:58.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_disabled_modules": "", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_enable_op_tracker": "true", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_max_pg_num_change": "128", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_mds_bytes": "134217728", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_mds_messages": "128", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_mon_bytes": "134217728", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_mon_messages": "128", 2024-09-15T00:08:58.441 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_history_duration": "600", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_history_size": "20", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_log_threshold": "5", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_osd_bytes": "536870912", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_osd_messages": "8192", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_pool": "true", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_standby_modules": "true", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_stats_period": "5", 2024-09-15T00:08:58.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_stats_threshold": "5", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_tick_period": "2", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_allow_pool_delete": "true", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_allow_pool_size_one": "true", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_auth_validate_all_caps": "true", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_bytes": "104857600", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_directed_command_retry": "5", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-15T00:08:58.443 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_parallel": "3", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_log_interval": "1.000000", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_ping_interval": "10.000000", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_target_rank": "-1", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_file_level": "debug", 2024-09-15T00:08:58.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_file": "true", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_journald": "false", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_compact_on_bootstrap": "false", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_compact_on_start": "false", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_compact_on_trim": "true", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-15T00:08:58.445 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cpu_threads": "4", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_crush_min_required_version": "hammer", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_daemon_bytes": "419430400", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_data_avail_crit": "5", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_data_avail_warn": "5", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_data_size_warn": "16106127360", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_dump_json": "false", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_dump_transactions": "false", 2024-09-15T00:08:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_extra_checks": "false", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_no_require_reef": "false", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_no_require_squid": "false", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_down_mkfs_grace": "120", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_down_uptime_grace": "60", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_election_default_strategy": "1", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_election_timeout": "5.000000", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_elector_ping_divisor": "2", 2024-09-15T00:08:58.447 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_enable_op_tracker": "true", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_fake_pool_delete": "false", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_force_quorum_join": "false", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_globalid_prealloc": "10000", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_detail_to_clog": "false", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_log_update_period": "5", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_max_detail": "50", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_to_clog": "true", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_to_clog_interval": "600", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_host": "172.21.15.27,[v2:172.21.15.27:3301,v1:172.21.15.27:6790],172.21.15.79", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_host_override": "", 2024-09-15T00:08:58.448 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_initial_members": "", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_lease": "5.000000", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_log_full_interval": "50", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_log_max": "10000", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_log_max_summary": "50", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-15T00:08:58.449 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_log_epochs": "500", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_osd": "10000", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_pg_per_osd": "10000", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_pool_pg_num": "65536", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mds_force_trim_to": "0", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mds_skip_sanity": "false", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_memory_autotune": "true", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_memory_target": "2147483648", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_beacon_grace": "30", 2024-09-15T00:08:58.450 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_digest_period": "5", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_inactive_grace": "60", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_min_osdmap_epochs": "500", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_complaint_time": "30", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_history_duration": "600", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_history_size": "20", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_history_slow_op_size": "20", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_log_threshold": "5", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-15T00:08:58.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_auto_mark_in": "false", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_cache_size": "500", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_down_out_interval": "600", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_force_trim_to": "0", 2024-09-15T00:08:58.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_min_down_reporters": "2", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-15T00:08:58.453 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_report_timeout": "900", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-15T00:08:58.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_stuck_threshold": "60", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_probe_timeout": "2.000000", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_reweight_max_change": "0.050000", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_reweight_max_osds": "4", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-15T00:08:58.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_interval": "86400", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_max_keys": "100", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_timeout": "300", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_session_timeout": "300", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_smart_report_timeout": "5", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stat_smooth_intervals": "6", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stretch_pool_min_size": "2", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stretch_pool_size": "4", 2024-09-15T00:08:58.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_debug": "false", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_provider_kill_at": "0", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_requester_kill_at": "0", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_timeout": "60.000000", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_target_pg_per_osd": "100", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_tick_interval": "5", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_timecheck_interval": "300.000000", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-15T00:08:58.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_use_min_delay_socket": "false", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_older_version_delay": "604800", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_misplaced": "false", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_older_version": "true", 2024-09-15T00:08:58.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "monmap": "", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-15T00:08:58.459 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_tracing_enabled": "false", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_op_threads": "3", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_cm": "false", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_device_name": "", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_dscp": "96", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_local_gid": "", 2024-09-15T00:08:58.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_port_num": "1", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_sl": "3", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_support_srq": "true", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_type": "ib", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_reap_threshold": "5", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_before_connect": "false", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_ipv4": "true", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_ipv6": "false", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_msgr1": "true", 2024-09-15T00:08:58.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_msgr2": "true", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_port_max": "7568", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_port_min": "6800", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_retry_count": "3", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_retry_delay": "5", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_client": "false", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_mds": "false", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_mgr": "false", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_mon": "false", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_osd": "false", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_client_mode": "crc secure", 2024-09-15T00:08:58.462 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_cluster_mode": "crc secure", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_cluster_type": "", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_compress_secure": "false", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_connection_idle_timeout": "900", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_connection_ready_timeout": "10", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_crc_data": "true", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_crc_header": "true", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_bad_msg": "false", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_bug": "true", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_old_message": "true", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_skipped_message": "false", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-15T00:08:58.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_coremask": "0xF", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_enable_tso": "true", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_hugepages": "", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_lro": "true", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_memory_channel": "4", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-15T00:08:58.464 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_pmd": "", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_port_id": "0", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dump_on_send": "false", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_initial_backoff": "0.200000", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_delay_max": "1.000000", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_delay_type": "", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_network_congestion": "0", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_socket_failures": "5000", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_learn_addr_from_peer": "true", 2024-09-15T00:08:58.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_max_accept_failures": "4", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_max_backoff": "15.000000", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_mon_client_mode": "secure crc", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_mon_service_mode": "secure crc", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_osd_compress_min_size": "1024", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_osd_compress_mode": "none", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_pq_min_cost": "65536", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_public_type": "", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_service_mode": "crc secure", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_tcp_listen_backlog": "512", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_tcp_nodelay": "true", 2024-09-15T00:08:58.466 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_tcp_rcvbuf": "0", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_type": "async+posix", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "no_config_file": "false", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_completion_locks_per_session": "32", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_inflight_ops": "1024", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_tick_interval": "5.000000", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_timeout": "10.000000", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objectstore_blackhole": "false", 2024-09-15T00:08:58.467 INFO:teuthology.orchestra.run.smithi027.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "openssl_engine_opts": "", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_delay_time": "5.000000", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_hist_halflife": "1000", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_max_low_ops": "2", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_max_ops": "4", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_slop": "0.020000", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-15T00:08:58.468 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_asio_thread_count": "2", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_async_recovery_min_cost": "100", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backfill_scan_max": "512", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backfill_scan_min": "64", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backoff_on_degraded": "false", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backoff_on_peering": "false", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backoff_on_unfound": "true", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_beacon_report_interval": "300", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bench_duration": "30", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bench_max_block_size": "67108864", 2024-09-15T00:08:58.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_blkin_trace_all": "false", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_check_for_log_corruption": "false", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_class_update_on_start": "true", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_client_message_cap": "256", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_client_message_size_cap": "524288000", 2024-09-15T00:08:58.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_client_op_priority": "63", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_client_watch_timeout": "30", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_command_max_records": "256", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_compact_on_start": "false", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_crush_update_on_start": "true", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_crush_update_weight_set": "true", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-15T00:08:58.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_feed_pullee": "-1", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_misdirected_ops": "false", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_no_acting_change": "false", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_no_purge_strays": "false", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_op_order": "false", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-15T00:08:58.472 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_shutdown": "false", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_trim_objects": "false", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_verify_snaps": "false", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_keys": "1024", 2024-09-15T00:08:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_stride": "524288", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_default_notify_timeout": "30", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_delete_sleep": "0.000000", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_discard_disconnected_ops": "true", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_enable_op_tracker": "true", 2024-09-15T00:08:58.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_info": "true", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_shutdown": "true", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_function_tracing": "false", 2024-09-15T00:08:58.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_grace": "20", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_interval": "6", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_min_peers": "10", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_min_size": "2000", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_stale": "600", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_hit_set_max_size": "100000", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_hit_set_min_size": "1000", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-15T00:08:58.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_journal_size": "5120", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_kill_backfill_at": "0", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_cache_size": "50", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_dedup": "true", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_message_max": "40", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_share_max_epochs": "40", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_attr_name_len": "100", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_attr_size": "0", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_backfills": "1", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_markdown_count": "5", 2024-09-15T00:08:58.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_markdown_period": "600", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_object_name_len": "2048", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_object_namespace_len": "256", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_object_size": "134217728", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_pg_blocked_by": "16", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_pg_log_entries": "10000", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_pgls": "1024", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_push_cost": "8388608", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_push_objects": "10", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_scrubs": "3", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-15T00:08:58.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_trimming_pgs": "2", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_write_op_reply_len": "64", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_write_size": "90", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_profile": "balanced", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-15T00:08:58.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_base": "805306368", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_cache_min": "134217728", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-15T00:08:58.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_target": "4294967296", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_target_autotune": "false", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_min_pg_log_entries": "250", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_min_recovery_priority": "0", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_report_interval": "5", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_num_cache_shards": "32", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_num_op_tracker_shard": "32", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_numa_auto_affinity": "true", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_numa_node": "-1", 2024-09-15T00:08:58.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_numa_prefer_iface": "true", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_objecter_finishers": "1", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_objectstore": "bluestore", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_objectstore_fuse": "false", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_objectstore_tracing": "false", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_complaint_time": "30.000000", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_history_duration": "600", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_history_size": "20", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_history_slow_op_size": "20", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_log_threshold": "5", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_shards": "0", 2024-09-15T00:08:58.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_shards_hdd": "5", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_shards_ssd": "8", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_pq_min_cost": "65536", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_queue_cut_off": "high", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_thread_timeout": "15", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_open_classes_on_start": "true", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_os_flags": "0", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_peering_op_priority": "255", 2024-09-15T00:08:58.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_delete_cost": "1048576", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_delete_priority": "5", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_log_trim_max": "10000", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_log_trim_min": "100", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-15T00:08:58.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_crimson": "false", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flags": "0", 2024-09-15T00:08:58.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_min_size": "0", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_pg_num": "32", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_pgp_num": "0", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_read_ratio": "70", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_size": "2", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_type": "replicated", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_push_per_object_cost": "1000", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-15T00:08:58.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recover_clone_overlap": "true", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_cost": "20971520", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_active": "0", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_single_start": "1", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_op_priority": "3", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_priority": "5", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-15T00:08:58.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_sleep": "0.000000", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_repair_during_recovery": "false", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_requested_scrub_priority": "120", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_auto_repair": "false", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_begin_hour": "0", 2024-09-15T00:08:58.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_begin_week_day": "0", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_chunk_max": "25", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_chunk_min": "5", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_cost": "52428800", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_during_recovery": "false", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_end_hour": "0", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_end_week_day": "0", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_event_cost": "4096", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_invalid_stats": "true", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-15T00:08:58.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_max_preemptions": "5", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_priority": "5", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_sleep": "0.000000", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_skip_data_digest": "false", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_smart_report_timeout": "5", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_cost": "1048576", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_priority": "5", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-15T00:08:58.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_target_transaction_size": "30", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-15T00:08:58.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tracing": "false", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_use_stale_snap": "false", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osdc_blkin_trace_all": "false", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_kill_at": "0", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_max_join_drift": "10", 2024-09-15T00:08:58.492 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_min": "500", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_min_wait": "0.050000", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_propose_interval": "1.000000", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_service_trim_max": "500", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_service_trim_min": "250", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_stash_full_interval": "25", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_trim_max": "500", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_trim_min": "250", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "perf": "true", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "-", 2024-09-15T00:08:58.493 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrv": "", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "public_bind_addr": "-", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "public_network": "", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "public_network_interface": "", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "qat_compressor_busy_polling": "false", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "qat_compressor_enabled": "false", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "qat_compressor_session_max_number": "256", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "rados_mon_op_timeout": "0", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "rados_osd_op_timeout": "0", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "rados_tracing": "false", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_atime_update_interval": "60", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_balance_parent_reads": "false", 2024-09-15T00:08:58.494 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_balance_snap_reads": "false", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_blkin_trace_all": "false", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache": "true", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_policy": "writearound", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_size": "33554432", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-15T00:08:58.495 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_clone_copy_on_read": "false", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_compression_hint": "none", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_concurrent_management_ops": "10", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_clone_format": "auto", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_data_pool": "", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_features": "61", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_format": "2", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_map_options": "", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_order": "22", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_pool": "rbd", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_stripe_count": "0", 2024-09-15T00:08:58.496 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_stripe_unit": "0", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_enable_alloc_hint": "true", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_io_scheduler": "simple", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-15T00:08:58.497 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_order": "24", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_pool": "", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_splay_width": "4", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_localize_parent_reads": "false", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_localize_snap_reads": "false", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-15T00:08:58.498 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-15T00:08:58.499 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mtime_update_interval": "60", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_non_blocking_aio": "true", 2024-09-15T00:08:58.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_op_thread_timeout": "60", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_op_threads": "1", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_parent_cache_enabled": "false", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_plugins": "", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_bps_burst": "0", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_bps_limit": "0", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_exclude_ops": "0", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_iops_burst": "0", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-15T00:08:58.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_iops_limit": "0", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-15T00:08:58.502 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_read_from_replica_policy": "default", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_skip_partial_discard": "true", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_tracing": "false", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_validate_names": "true", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_validate_pool": "true", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "restapi_base_url": "", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "restapi_log_level": "", 2024-09-15T00:08:58.503 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_acl_grants_max_num": "100", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_admin_entry": "admin", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_backend_store": "rados", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_barbican_url": "", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_beast_enable_async": "true", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_counters_cache": "false", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-15T00:08:58.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-15T00:08:58.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-15T00:08:58.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-15T00:08:58.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-15T00:08:58.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cache_enabled": "true", 2024-09-15T00:08:58.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cache_expiry_interval": "900", 2024-09-15T00:08:58.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cache_lru_size": "25000", 2024-09-15T00:08:58.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_config_store": "rados", 2024-09-15T00:08:58.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_content_length_compat": "false", 2024-09-15T00:08:58.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_copy_obj_progress": "true", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cors_rules_max_num": "100", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cross_domain_policy": "", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_addr": "", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_password": "", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-15T00:08:58.508 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_username": "", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_require_ssl": "true", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-15T00:08:58.509 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_addr": "", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_auth": "token", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_namespace": "", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_prefix": "", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_token_file": "", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-15T00:08:58.510 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_buffersize": "524288", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_low_speed_time": "300", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-15T00:08:58.511 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_port": "6379", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_log_changes_size": "1000", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_log_num_shards": "128", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_log_window": "30", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-15T00:08:58.512 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-15T00:08:58.513 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dns_name": "", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dns_s3website_name": "", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dynamic_resharding": "true", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_gc_threads": "true", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_lc_threads": "true", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_ops_log": "false", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_quota_threads": "true", 2024-09-15T00:08:58.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_static_website": "false", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_usage_log": "false", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enforce_swift_acls": "true", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_exit_timeout_secs": "120", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_expose_bucket": "false", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_extended_http_attrs": "", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_filter": "none", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_frontends": "beast port=7480", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_deferred": "50", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_objs": "32", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-15T00:08:58.515 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_processor_period": "3600", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_init_timeout": "300", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_kafka_connection_idle": "300", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-15T00:08:58.516 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_domain": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_password": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_password_path": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_project": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_tenant": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_token": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_token_path": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_user": "", 2024-09-15T00:08:58.517 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_api_version": "2", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_domain": "", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_password": "", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_project": "", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_user": "", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_url": "", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_debug_interval": "-1", 2024-09-15T00:08:58.518 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_lock_max_time": "90", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_max_objs": "32", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_max_rules": "1000", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_max_worker": "3", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_thread_delay": "0", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_dnattr": "uid", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_searchfilter": "", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_uri": "", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-15T00:08:58.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_log_http_headers": "", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_log_object_name_utc": "false", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_attr_name_len": "0", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_attr_size": "0", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_chunk_size": "4194304", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-15T00:08:58.520 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_listing_results": "1000", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_notify_retries": "10", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_put_param_size": "1048576", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_put_size": "5368709120", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_slo_entries": "1000", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_md_log_max_shards": "64", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-15T00:08:58.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_mp_lock_max_time": "600", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_max_gc": "300", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-15T00:08:58.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_num_async_rados_threads": "32", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_num_control_oids": "8", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_numa_node": "-1", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_objexp_chunk_size": "100", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_objexp_gc_interval": "600", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-15T00:08:58.523 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_op_thread_timeout": "600", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_op_tracing": "false", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_opa_token": "", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_opa_url": "", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_opa_verify_ssl": "true", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ops_log_rados": "false", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ops_log_socket_path": "", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_period_push_interval": "2.000000", 2024-09-15T00:08:58.524 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_cache_lanes": "3", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_cache_partitions": "3", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_print_continue": "true", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-15T00:08:58.525 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_rados_tracing": "false", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_realm": "", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_realm_id": "", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_region": "", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_request_uri": "", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_batch_size": "64", 2024-09-15T00:08:58.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_max_aio": "128", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_num_logs": "16", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_thread_interval": "600", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_resolve_cname": "false", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_run_sync_thread": "true", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-15T00:08:58.527 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_scheduler_type": "throttler", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_script_uri": "", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_service_provider_name": "", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_client_id": "", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_client_secret": "", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_entry": "sts", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_key": "sts", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_min_session_duration": "900", 2024-09-15T00:08:58.528 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_token_introspection_url": "", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_account_in_url": "false", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_auth_entry": "auth", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_auth_url": "", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_custom_header": "", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_need_stats": "true", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_tenant_name": "", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_token_expiration": "86400", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_url": "", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_url_prefix": "swift", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-15T00:08:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_lease_period": "120", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_thread_pool_size": "512", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-15T00:08:58.530 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_comment": "", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_createby": "", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_encoding": "", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_flag": "false", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_origin": "", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_tracker": "", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_trust_forwarded_https": "false", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_usage_max_shards": "32", 2024-09-15T00:08:58.531 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_usage_max_user_shards": "1", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_use_opa_authz": "false", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_counters_cache": "false", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_max_buckets": "1000", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_policies_max_num": "100", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_unique_email": "true", 2024-09-15T00:08:58.532 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_verify_ssl": "true", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zone": "", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zone_id": "", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zonegroup": "", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zonegroup_id": "", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rgwlc_auto_session_clear": "true", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_block_size": "4096", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-15T00:08:58.533 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_size": "536870912", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_index_type": "binary_search", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-15T00:08:58.534 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_partition_filters": "false", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_perf": "false", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "run_dir": "/var/run/ceph", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_block_create": "true", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_cache_lru_size": "67108864", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_default_max_object_size": "16777216", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-15T00:08:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_device_size": "53687091200", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_full_integrity_check": "false", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_journal_batch_capacity": "16", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_main_device_type": "SSD", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-15T00:08:58.536 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_segment_size": "67108864", 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout: "set_keepcaps": "false", 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout: "setgroup": "", 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout: "setuser": "", 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout: "setuser_match_path": "", 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout: "thp": "false", 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout: "threadpool_default_timeout": "60", 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout: "throttler_perf_counter": "true" 2024-09-15T00:08:58.537 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:08:58.539 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-15T00:08:58.655 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a: {} 2024-09-15T00:08:58.655 INFO:teuthology.orchestra.run.smithi027.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:08:58.664 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b: {} 2024-09-15T00:08:58.664 INFO:teuthology.orchestra.run.smithi027.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:08:58.673 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c: {} 2024-09-15T00:08:58.673 INFO:teuthology.orchestra.run.smithi027.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:08:58.684 DEBUG:teuthology.orchestra.run.smithi079:> type ceph-objectstore-tool 2024-09-15T00:08:58.702 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T00:08:58.702 DEBUG:teuthology.orchestra.run.smithi027:> type ceph-objectstore-tool 2024-09-15T00:08:58.717 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T00:08:58.717 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-15T00:08:58.730 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-15T00:08:58.730 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-15T00:08:58.730 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-15T00:08:58.731 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:08:58.731 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T00:08:58.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:08:58.731 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-15T00:08:58.732 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-15T00:08:58.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:08:58.732 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-15T00:08:58.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:08:58.733 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-15T00:08:58.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:08:58.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-15T00:08:58.760 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:08:58.757+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:08:58.829 INFO:teuthology.orchestra.run.smithi092.stdout:mapped writes DISABLED 2024-09-15T00:08:58.830 INFO:teuthology.orchestra.run.smithi092.stdout:mapped reads DISABLED 2024-09-15T00:08:58.830 INFO:teuthology.orchestra.run.smithi092.stdout:Seed set to 1413 2024-09-15T00:08:58.830 INFO:teuthology.orchestra.run.smithi092.stdout:rbd-nbd mode enabled 2024-09-15T00:08:58.833 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:08:58.831+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:08:58.876 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:08:58.876 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:08:58.877 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:08:58.877 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:08:58.887 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:08:58.897 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:08:58.897 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:08:58.920 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:08:58.920 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:08:58.933 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:08:58.931+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:08:58.943 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:08:58.943 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:08:58.977 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:08:58.977 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:08:58.994 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:08:58.994 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:08:59.004 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:08:59.004 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:08:59.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:08:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:08:59.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:08:59.065 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:08:59.063+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:08:59.112 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:08:59.112 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:08:59.112 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 20, 2024-09-15T00:08:59.112 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "4f532d31-28c1-4236-a74e-6392960baf2b", 2024-09-15T00:08:59.112 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2024-09-15T00:08:32.007191+0000", 2024-09-15T00:08:59.112 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2024-09-15T00:08:58.929182+0000", 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2024-09-15T00:08:42.794810+0000", 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2024-09-15T00:08:33.890963+0000", 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2024-09-15T00:08:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 3, 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 6, 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "squid", 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2024-09-15T00:08:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:42.896076+0000", 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:08:59.115 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:08:59.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "14", 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:08:59.117 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:08:59.118 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:08:59.119 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:08:59.120 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:08:59.120 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:08:59.120 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:08:59.120 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:08:59.120 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2024-09-15T00:08:59.120 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2024-09-15T00:08:59.121 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2024-09-15T00:08:59.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.121 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:08:59.121 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2024-09-15T00:08:59.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.121 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:08:59.122 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:08:59.122 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 6.059999942779541, 2024-09-15T00:08:59.122 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 6.059999942779541, 2024-09-15T00:08:59.122 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.33000001311302185, 2024-09-15T00:08:59.122 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T00:08:59.122 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T00:08:59.122 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:08:59.123 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:08:59.123 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:08:59.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.123 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 2, 2024-09-15T00:08:59.123 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "rbd", 2024-09-15T00:08:59.124 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:49.173060+0000", 2024-09-15T00:08:59.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:08:59.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:08:59.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:08:59.124 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:08:59.124 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:08:59.124 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:08:59.125 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:08:59.125 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:08:59.125 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:08:59.125 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:08:59.125 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:08:59.125 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:08:59.126 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:08:59.126 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 8, 2024-09-15T00:08:59.126 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 8, 2024-09-15T00:08:59.126 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 8, 2024-09-15T00:08:59.126 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 8, 2024-09-15T00:08:59.127 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 8, 2024-09-15T00:08:59.127 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:08:59.127 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:08:59.127 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:08:59.127 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:08:59.127 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "18", 2024-09-15T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:08:59.128 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 18, 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:08:59.129 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:08:59.130 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:08:59.131 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "pool_client.0", 2024-09-15T00:08:59.132 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:58.879058+0000", 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 32769, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,creating", 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:08:59.133 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:08:59.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "20", 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:08:59.135 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:08:59.136 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:08:59.137 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:08:59.138 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:08:59.138 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:08:59.138 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:08:59.138 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:08:59.138 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:08:59.138 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:08:59.138 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": {}, 2024-09-15T00:08:59.138 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:08:59.138 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:08:59.139 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2024-09-15T00:08:59.139 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2024-09-15T00:08:59.139 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:08:59.139 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2024-09-15T00:08:59.139 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2024-09-15T00:08:59.139 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:08:59.139 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:08:59.139 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:08:59.140 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.140 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.140 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:08:59.140 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T00:08:59.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.140 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:08:59.140 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "56600dc2-b95d-47a7-b2b1-5ddc54c08341", 2024-09-15T00:08:59.140 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:08:59.141 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:08:59.141 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:08:59.141 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:08:59.141 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:08:59.141 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:08:59.141 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:08:59.141 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 0, 2024-09-15T00:08:59.141 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:08:59.142 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:08:59.142 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:08:59.142 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.142 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2024-09-15T00:08:59.142 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:08:59.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.143 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2024-09-15T00:08:59.143 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:08:59.143 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.143 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.143 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:08:59.144 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.144 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2024-09-15T00:08:59.144 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:08:59.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.145 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2024-09-15T00:08:59.145 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:08:59.145 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.145 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.145 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:08:59.145 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.146 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2024-09-15T00:08:59.146 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:08:59.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.146 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2024-09-15T00:08:59.146 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:08:59.147 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.147 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.147 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:08:59.147 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.147 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2024-09-15T00:08:59.148 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:08:59.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.148 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.148 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2024-09-15T00:08:59.148 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:08:59.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.148 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.149 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/1440638768", 2024-09-15T00:08:59.149 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/1440638768", 2024-09-15T00:08:59.149 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6823/1440638768", 2024-09-15T00:08:59.149 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6821/1440638768", 2024-09-15T00:08:59.149 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:08:59.149 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:08:59.149 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:08:59.150 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.150 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:08:59.150 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "ea8f0155-8472-4e87-89fd-7716d05065d0", 2024-09-15T00:08:59.150 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:08:59.150 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:08:59.150 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 15, 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:08:59.152 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.153 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/3601510193", 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/3601510193", 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/3601510193", 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/3601510193", 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "357d2d59-66f0-47b8-87c8-4acdfe62a07d", 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:08:59.156 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 15, 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.157 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.160 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.161 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/3222759256", 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/3222759256", 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/3222759256", 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/3222759256", 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.162 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "22208f56-d8aa-4a56-a42c-5a990962b860", 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 15, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:08:59.163 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6809", 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6811", 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6813", 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6816", 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.165 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6822", 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6823", 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:08:59.166 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6818", 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.167 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6821", 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6811/675428777", 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6816/675428777", 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6823/675428777", 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6821/675428777", 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "a974629e-9ef0-40a3-94bd-382e0bfa8bb6", 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 0, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:08:59.169 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6808", 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6810", 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:08:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6812", 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6814", 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6819", 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6820", 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6815", 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:08:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6817", 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6810/3736884138", 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6814/3736884138", 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6820/3736884138", 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6817/3736884138", 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:08:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "06b73c2d-466a-4c4e-89fa-31d6eeb1af1d", 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:08:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 15, 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6800", 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6801", 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6802", 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6803", 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6806", 2024-09-15T00:08:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6807", 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6804", 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6805", 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6801/2235671634", 2024-09-15T00:08:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6803/2235671634", 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6807/2235671634", 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6805/2235671634", 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:08:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.730107+0000", 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:08:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.365297+0000", 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.868500+0000", 2024-09-15T00:08:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.788781+0000", 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.892143+0000", 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:08:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.990037+0000", 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2024-09-15T00:08:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2024-09-15T00:08:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2024-09-15T00:08:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:08:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:08:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2024-09-15T00:08:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:08:59.188 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:08:59.190 INFO:tasks.thrashosds.thrasher:j is {'epoch': 20, 'fsid': '4f532d31-28c1-4236-a74e-6392960baf2b', 'created': '2024-09-15T00:08:32.007191+0000', 'modified': '2024-09-15T00:08:58.929182+0000', 'last_up_change': '2024-09-15T00:08:42.794810+0000', 'last_in_change': '2024-09-15T00:08:33.890963+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:08:42.896076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:08:49.173060+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:08:58.879058+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '56600dc2-b95d-47a7-b2b1-5ddc54c08341', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 1440638768}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 1440638768}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 1440638768}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 1440638768}]}, 'public_addr': '172.21.15.27:6817/1440638768', 'cluster_addr': '172.21.15.27:6819/1440638768', 'heartbeat_back_addr': '172.21.15.27:6823/1440638768', 'heartbeat_front_addr': '172.21.15.27:6821/1440638768', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ea8f0155-8472-4e87-89fd-7716d05065d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 3601510193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 3601510193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 3601510193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 3601510193}]}, 'public_addr': '172.21.15.27:6809/3601510193', 'cluster_addr': '172.21.15.27:6811/3601510193', 'heartbeat_back_addr': '172.21.15.27:6815/3601510193', 'heartbeat_front_addr': '172.21.15.27:6813/3601510193', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '357d2d59-66f0-47b8-87c8-4acdfe62a07d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 3222759256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 3222759256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 3222759256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 3222759256}]}, 'public_addr': '172.21.15.27:6801/3222759256', 'cluster_addr': '172.21.15.27:6803/3222759256', 'heartbeat_back_addr': '172.21.15.27:6807/3222759256', 'heartbeat_front_addr': '172.21.15.27:6805/3222759256', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '22208f56-d8aa-4a56-a42c-5a990962b860', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6809', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 675428777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6813', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6816', 'nonce': 675428777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 675428777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 675428777}]}, 'public_addr': '172.21.15.79:6811/675428777', 'cluster_addr': '172.21.15.79:6816/675428777', 'heartbeat_back_addr': '172.21.15.79:6823/675428777', 'heartbeat_front_addr': '172.21.15.79:6821/675428777', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a974629e-9ef0-40a3-94bd-382e0bfa8bb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6810', 'nonce': 3736884138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6814', 'nonce': 3736884138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6819', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6820', 'nonce': 3736884138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6815', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 3736884138}]}, 'public_addr': '172.21.15.79:6810/3736884138', 'cluster_addr': '172.21.15.79:6814/3736884138', 'heartbeat_back_addr': '172.21.15.79:6820/3736884138', 'heartbeat_front_addr': '172.21.15.79:6817/3736884138', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '06b73c2d-466a-4c4e-89fa-31d6eeb1af1d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 2235671634}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2235671634}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2235671634}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2235671634}]}, 'public_addr': '172.21.15.79:6801/2235671634', 'cluster_addr': '172.21.15.79:6803/2235671634', 'heartbeat_back_addr': '172.21.15.79:6807/2235671634', 'heartbeat_front_addr': '172.21.15.79:6805/2235671634', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.730107+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.365297+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.868500+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.788781+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.892143+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.990037+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:08:59.190 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-15T00:08:59.191 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:08:59.191 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:08:59.192 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:08:59.192 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:08:59.187+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:08:59.290 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:08:59.288+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:08:59.338 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:08:59.338 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:08:59.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:08:59.392 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:08:59.391+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:08:59.493 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:08:59.492+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:08:59.493 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:08:59.494 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:08:59.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:08:59.594 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:08:59.592+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:08:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:08:59.609 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:08:59.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:08:59.694 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:08:59.693+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:08:59.766 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:08:59.766 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:08:59.774 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:08:59.819 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:08:59.817+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:08:59.881 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:08:59.881 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:08:59.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:08:59.931 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:08:59.929+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:08:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:08:59.997 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:08:59.995+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:00.046 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:00.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:00.099 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:00.098+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:00.170 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:00.170 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:00.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:00.199 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:00.197+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:00.300 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:00.298+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:00.324 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:00.324 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:00.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:00.416 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:00.414+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:00.475 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:00.475 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:00.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:00.525 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:00.523+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:00.602 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:00.600+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:00.626 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:00.627 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:00.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:00.703 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:00.702+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:00.737 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:00.737 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:00.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:00.804 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:00.802+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:00.847 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:00.848 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:00.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:00.905 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:00.903+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:00.960 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:00.960 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:00.969 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:01.006 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:01.004+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:01.024 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:01.071 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:01.071 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:01.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:01.107 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:01.105+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:01.157 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:01.157 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:01.174 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:01.174 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:01.181 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:01.182 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:01.190 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:01.191 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:01.191 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:01.207 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:01.208 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:01.205+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:01.224 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:01.224 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:01.242 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:01.299 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:01.300 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:01.300 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:01.300 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:01.308 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:01.310 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:01.308+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:01.410 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:01.409+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:01.416 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:01.416 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:01.416 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:01.416 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:01.416 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:01.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:01.511 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:01.509+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:01.533 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:01.533 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:01.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:01.533 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:01.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.533 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:44.868509+0000", 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.662517394999998, 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053926299999999998, 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.868509+0000", 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.868509+0000", 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.868509+0000", 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5899999999999996e-07 2024-09-15T00:09:01.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.868589+0000", 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9689000000000002e-05 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.868599+0000", 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0197000000000001e-05 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.868609+0000", 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0258000000000001e-05 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.868646+0000", 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6878000000000002e-05 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.868659+0000", 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2982999999999999e-05 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.873866+0000", 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052068660000000001 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.873901+0000", 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5200000000000002e-05 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:44.869046+0000", 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.661979890000001, 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0048726300000000002, 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.869046+0000", 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.869046+0000", 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.869047+0000", 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.58e-07 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.869060+0000", 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3723999999999999e-05 2024-09-15T00:09:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.869069+0000", 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2879999999999993e-06 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.869072+0000", 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8419999999999998e-06 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.869110+0000", 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7608999999999999e-05 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.869121+0000", 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1467999999999999e-05 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.873903+0000", 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047813029999999998 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.873919+0000", 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5937999999999999e-05 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:58 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'36, mlcod=13'33)", 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:44.875264+0000", 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.655762494000001, 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039460049999999998, 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.543 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875264+0000", 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875264+0000", 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.544 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875268+0000", 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2830000000000003e-06 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875342+0000", 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4300000000000004e-05 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875351+0000", 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9190000000000005e-06 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875355+0000", 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2710000000000003e-06 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875510+0000", 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015459299999999999 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875539+0000", 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9522e-05 2024-09-15T00:09:01.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.879173+0000", 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0036338030000000001 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.879210+0000", 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6313999999999998e-05 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:59 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'37, mlcod=13'33)", 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:44.875709+0000", 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.655317168, 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035191319999999999, 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875709+0000", 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875709+0000", 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875709+0000", 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7700000000000004e-07 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875716+0000", 2024-09-15T00:09:01.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8820000000000003e-06 2024-09-15T00:09:01.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.550 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875729+0000", 2024-09-15T00:09:01.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2628e-05 2024-09-15T00:09:01.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875733+0000", 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.138e-06 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875804+0000", 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0810999999999997e-05 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.875816+0000", 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2577e-05 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.879212+0000", 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0033953880000000001 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:44.879228+0000", 2024-09-15T00:09:01.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6130999999999999e-05 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:109 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'71, mlcod=13'64)", 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.064696+0000", 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.466330107000001, 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053589299999999996, 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.064696+0000", 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.064696+0000", 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.064696+0000", 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4999999999999998e-07 2024-09-15T00:09:01.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.064753+0000", 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6273999999999997e-05 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.064760+0000", 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6199999999999999e-06 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.064764+0000", 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7340000000000002e-06 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.064796+0000", 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2348999999999998e-05 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.064808+0000", 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1773999999999999e-05 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070023+0000", 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052152989999999996 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070055+0000", 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1430000000000002e-05 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:110 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'72, mlcod=13'67)", 2024-09-15T00:09:01.557 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.065494+0000", 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.465531682000002, 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0045897400000000001, 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065494+0000", 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065494+0000", 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065495+0000", 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065500+0000", 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9060000000000001e-06 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065508+0000", 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.943e-06 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065512+0000", 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0210000000000001e-06 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065547+0000", 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5573e-05 2024-09-15T00:09:01.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065559+0000", 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.132e-05 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070056+0000", 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0044977230000000003 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070084+0000", 2024-09-15T00:09:01.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7783000000000001e-05 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:111 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'73, mlcod=13'67)", 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.065908+0000", 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.465117512999999, 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042038539999999999, 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065908+0000", 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065908+0000", 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065909+0000", 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5700000000000002e-07 2024-09-15T00:09:01.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065915+0000", 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2049999999999999e-06 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065925+0000", 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8439999999999993e-06 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065932+0000", 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8050000000000001e-06 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065968+0000", 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6115000000000002e-05 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.065978+0000", 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0154e-05 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070085+0000", 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00410722 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070112+0000", 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6954e-05 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.566 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:112 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'74, mlcod=13'67)", 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.066249+0000", 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.464777132999998, 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038876370000000002, 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066249+0000", 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066249+0000", 2024-09-15T00:09:01.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066249+0000", 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2300000000000001e-07 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066297+0000", 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7754999999999999e-05 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066308+0000", 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0403e-05 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066312+0000", 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.741e-06 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066347+0000", 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4484999999999999e-05 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066359+0000", 2024-09-15T00:09:01.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1823e-05 2024-09-15T00:09:01.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070113+0000", 2024-09-15T00:09:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0037547100000000001 2024-09-15T00:09:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070137+0000", 2024-09-15T00:09:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3096999999999999e-05 2024-09-15T00:09:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:113 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'75, mlcod=13'67)", 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.066694+0000", 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.464332388999999, 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034729090000000002, 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066694+0000", 2024-09-15T00:09:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066694+0000", 2024-09-15T00:09:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066694+0000", 2024-09-15T00:09:01.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000002e-07 2024-09-15T00:09:01.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066699+0000", 2024-09-15T00:09:01.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1429999999999997e-06 2024-09-15T00:09:01.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.577 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.577 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066706+0000", 2024-09-15T00:09:01.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9650000000000002e-06 2024-09-15T00:09:01.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066710+0000", 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.693e-06 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066743+0000", 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3510000000000003e-05 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.066754+0000", 2024-09-15T00:09:01.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0545e-05 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070139+0000", 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003384897 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.070167+0000", 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7690000000000001e-05 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:129 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'87, mlcod=13'85)", 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.372867+0000", 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.158159154, 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040562910000000001, 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.372867+0000", 2024-09-15T00:09:01.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.372867+0000", 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.372867+0000", 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3e-07 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.372872+0000", 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0010000000000001e-06 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.372882+0000", 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9010000000000006e-06 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.372887+0000", 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4229999999999998e-06 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.372953+0000", 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6234000000000003e-05 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.372966+0000", 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2843000000000001e-05 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.376898+0000", 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003932428 2024-09-15T00:09:01.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.376923+0000", 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4831e-05 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:130 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'88, mlcod=13'85)", 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.373259+0000", 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.157767311000001, 2024-09-15T00:09:01.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0036900779999999998, 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.373259+0000", 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.373259+0000", 2024-09-15T00:09:01.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.373259+0000", 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.373264+0000", 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3549999999999998e-06 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.373272+0000", 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5490000000000006e-06 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.373276+0000", 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0949999999999998e-06 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.373322+0000", 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5738000000000002e-05 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.373336+0000", 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3653e-05 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.376926+0000", 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035905870000000001 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.376949+0000", 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2560000000000001e-05 2024-09-15T00:09:01.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'85)", 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.374903+0000", 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.156123255000001, 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042045900000000002, 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.374903+0000", 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.374903+0000", 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.374904+0000", 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2500000000000001e-06 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.374984+0000", 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9814999999999994e-05 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.374991+0000", 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8240000000000002e-06 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.374995+0000", 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:09:01.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375062+0000", 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7168000000000005e-05 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375076+0000", 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3784000000000001e-05 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.379087+0000", 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040106179999999996 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.379107+0000", 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0611e-05 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:135 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'93, mlcod=13'85)", 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.375374+0000", 2024-09-15T00:09:01.593 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.155651730999999, 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0037458779999999998, 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375374+0000", 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375374+0000", 2024-09-15T00:09:01.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375375+0000", 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375378+0000", 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3579999999999999e-06 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375385+0000", 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5400000000000001e-06 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375388+0000", 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0790000000000001e-06 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.376088+0000", 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070034399999999999 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.376098+0000", 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0436e-05 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.379108+0000", 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0030098730000000001 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.379120+0000", 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1827999999999999e-05 2024-09-15T00:09:01.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:136 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'94, mlcod=13'85)", 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.375744+0000", 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.15528226, 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003689562, 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375744+0000", 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375744+0000", 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375744+0000", 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1999999999999999e-07 2024-09-15T00:09:01.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375749+0000", 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5379999999999996e-06 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375759+0000", 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0027000000000001e-05 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.375763+0000", 2024-09-15T00:09:01.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2710000000000003e-06 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.376312+0000", 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00054863100000000005 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.376341+0000", 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9385000000000001e-05 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.379414+0000", 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0030730789999999998 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.379433+0000", 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9011000000000001e-05 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:143 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'101, mlcod=13'98)", 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.380738+0000", 2024-09-15T00:09:01.602 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.150288335999999, 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004738969, 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.380738+0000", 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.380738+0000", 2024-09-15T00:09:01.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.380738+0000", 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.380805+0000", 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6916999999999996e-05 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.380813+0000", 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8189999999999992e-06 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.380817+0000", 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.873e-06 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.380851+0000", 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4069000000000002e-05 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.380863+0000", 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1789e-05 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.385448+0000", 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0045857099999999998 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.385477+0000", 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8289999999999998e-05 2024-09-15T00:09:01.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:144 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'102, mlcod=13'98)", 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.381113+0000", 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.149912821000001, 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0043791949999999998, 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381113+0000", 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381113+0000", 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381114+0000", 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2799999999999996e-07 2024-09-15T00:09:01.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381194+0000", 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0457999999999994e-05 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381201+0000", 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9619999999999996e-06 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381205+0000", 2024-09-15T00:09:01.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6210000000000001e-06 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381238+0000", 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3297999999999999e-05 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381249+0000", 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1307e-05 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.385479+0000", 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042297970000000004 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.385492+0000", 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3224000000000001e-05 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:145 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'103, mlcod=13'98)", 2024-09-15T00:09:01.611 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.381481+0000", 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.149545379999999, 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040343369999999998, 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381481+0000", 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381481+0000", 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381481+0000", 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2200000000000004e-07 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381574+0000", 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3327999999999998e-05 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381582+0000", 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1770000000000001e-06 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381601+0000", 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8992e-05 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381634+0000", 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3667999999999997e-05 2024-09-15T00:09:01.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.381646+0000", 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.149e-05 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.385493+0000", 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038473169999999998 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.385515+0000", 2024-09-15T00:09:01.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1843000000000001e-05 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:158 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'112, mlcod=13'105)", 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.800117+0000", 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.730909372999999, 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052327329999999998, 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.616 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800117+0000", 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800117+0000", 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800117+0000", 2024-09-15T00:09:01.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800179+0000", 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1645999999999999e-05 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800190+0000", 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1229e-05 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800196+0000", 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2519999999999996e-06 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800241+0000", 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5209e-05 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800254+0000", 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2862e-05 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.805321+0000", 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050667969999999996 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.805349+0000", 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8218999999999999e-05 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.620 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:159 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'113, mlcod=13'107)", 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.800695+0000", 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.730331327, 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046699250000000001, 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800695+0000", 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800695+0000", 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800695+0000", 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9500000000000003e-07 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800699+0000", 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8249999999999998e-06 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800708+0000", 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6279999999999994e-06 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800712+0000", 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2289999999999997e-06 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800750+0000", 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8257000000000003e-05 2024-09-15T00:09:01.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.800763+0000", 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3382e-05 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.805351+0000", 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0045877210000000003 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.805365+0000", 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3388e-05 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4174.0:163 1.0 e14/12 1:30306672:devicehealth::main.db.0000000000000000:head v 14'117, mlcod=13'115)", 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:45.811353+0000", 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.719673289999999, 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0053472839999999999, 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:01.625 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.811353+0000", 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.811353+0000", 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.811353+0000", 2024-09-15T00:09:01.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8500000000000002e-07 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.811357+0000", 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4199999999999999e-06 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.811365+0000", 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6200000000000005e-06 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:01.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.811372+0000", 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4169999999999997e-06 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.811424+0000", 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2488999999999998e-05 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.811443+0000", 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8991000000000002e-05 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.816676+0000", 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052325339999999996 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:45.816700+0000", 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4328000000000002e-05 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:01.629 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:01.631 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:01.631 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:01.629+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:01.732 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:01.730+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:01.779 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:01.780 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:01.788 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:01.872 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:01.870+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:01.934 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:01.932+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:01.935 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:01.935 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:01.935 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:01.935 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:01.935 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:01.944 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:01.956 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:09:02.035 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:02.033+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4208.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:51.841030+0000", 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.253395431, 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010715919999999999, 2024-09-15T00:09:02.097 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.841030+0000", 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.841030+0000", 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.841030+0000", 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9999999999999997e-07 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.841052+0000", 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1769000000000001e-05 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.841099+0000", 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6455999999999997e-05 2024-09-15T00:09:02.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.841120+0000", 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1651999999999999e-05 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.841255+0000", 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000134781 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.841283+0000", 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7823000000000001e-05 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.842051+0000", 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00076785700000000004 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:51.842101+0000", 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0553999999999997e-05 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4208.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:52.839654+0000", 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.2547712369999999, 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012324860000000001, 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.839654+0000", 2024-09-15T00:09:02.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.839654+0000", 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.839654+0000", 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2300000000000002e-07 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.839667+0000", 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2455e-05 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.839681+0000", 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4477999999999999e-05 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.839694+0000", 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3227999999999999e-05 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.839779+0000", 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4907999999999994e-05 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.839808+0000", 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9047000000000002e-05 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.840818+0000", 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001009818 2024-09-15T00:09:02.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:52.840886+0000", 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8229e-05 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4393.0:9 3.1a e23/20 3:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.960747+0000", 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.13367833800000001, 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00063275799999999997, 2024-09-15T00:09:02.106 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.960747+0000", 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.960747+0000", 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.960747+0000", 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7499999999999999e-07 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.960751+0000", 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.827e-06 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.960760+0000", 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8680000000000005e-06 2024-09-15T00:09:02.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.960766+0000", 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5950000000000004e-06 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.960808+0000", 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2188999999999999e-05 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.960824+0000", 2024-09-15T00:09:02.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5537e-05 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.961355+0000", 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053119099999999998 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.961380+0000", 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4375999999999998e-05 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:12 3.4 3:20776621:::rbd_header.11291357fc5b:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.976491+0000", 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.11793440500000001, 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013708240000000001, 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 12 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976491+0000", 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976491+0000", 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976492+0000", 2024-09-15T00:09:02.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2559999999999999e-06 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976496+0000", 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2649999999999998e-06 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976504+0000", 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9880000000000004e-06 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976511+0000", 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3040000000000001e-06 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976557+0000", 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6301000000000003e-05 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976656+0000", 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8709999999999995e-05 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976795+0000", 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013950900000000001 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977411+0000", 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061597900000000001 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977765+0000", 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00035408800000000002 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977819+0000", 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4101999999999998e-05 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977862+0000", 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2321999999999999e-05 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop_reply(client.4393.0:12 3.4 e22/20 ondisk, result = 0)", 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.977676+0000", 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.116748936, 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000190237, 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977676+0000", 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977676+0000", 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977677+0000", 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0200000000000001e-07 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977681+0000", 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1699999999999999e-06 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977694+0000", 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2451e-05 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977703+0000", 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1020000000000003e-06 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977750+0000", 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7243999999999997e-05 2024-09-15T00:09:02.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977758+0000", 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9880000000000004e-06 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977866+0000", 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010857999999999999 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:16 3.4 3:20776621:::rbd_header.11291357fc5b:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.985283+0000", 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.109142353, 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026770400000000001, 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 16 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.121 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985283+0000", 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985283+0000", 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985284+0000", 2024-09-15T00:09:02.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8299999999999995e-07 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985289+0000", 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0719999999999999e-06 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985299+0000", 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0189e-05 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985304+0000", 2024-09-15T00:09:02.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1050000000000003e-06 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985345+0000", 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0951999999999997e-05 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985404+0000", 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9011000000000001e-05 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985550+0000", 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014649199999999999 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:17 3.4 3:20776621:::rbd_header.11291357fc5b:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.985898+0000", 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.10852741, 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026059999999999999, 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 17 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985898+0000", 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985898+0000", 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985898+0000", 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7000000000000004e-07 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985902+0000", 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8550000000000004e-06 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985912+0000", 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3049999999999998e-06 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985920+0000", 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3059999999999998e-06 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.985966+0000", 2024-09-15T00:09:02.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6056000000000001e-05 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986028+0000", 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2365000000000003e-05 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986158+0000", 2024-09-15T00:09:02.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013004299999999999 2024-09-15T00:09:02.130 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.130 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.130 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.130 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:18 3.4 3:20776621:::rbd_header.11291357fc5b:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.130 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.986350+0000", 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.10807499399999999, 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000184, 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 18 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986350+0000", 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986350+0000", 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986351+0000", 2024-09-15T00:09:02.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986355+0000", 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1629999999999998e-06 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986363+0000", 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6389999999999998e-06 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986368+0000", 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1269999999999996e-06 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986410+0000", 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2802000000000001e-05 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986451+0000", 2024-09-15T00:09:02.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0182000000000003e-05 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986534+0000", 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3553000000000005e-05 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:19 3.4 3:20776621:::rbd_header.11291357fc5b:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.986710+0000", 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.10771562699999999, 2024-09-15T00:09:02.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015878800000000001, 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 19 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986710+0000", 2024-09-15T00:09:02.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986710+0000", 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986710+0000", 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8699999999999995e-07 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986714+0000", 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.352e-06 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986722+0000", 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8229999999999995e-06 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986726+0000", 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1409999999999998e-06 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986760+0000", 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3559000000000002e-05 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986797+0000", 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7404999999999998e-05 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.986868+0000", 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0920999999999994e-05 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.139 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20 3.4 3:20776621:::rbd_header.11291357fc5b:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.987307+0000", 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.107117978, 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00035387800000000002, 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.987307+0000", 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.987307+0000", 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.987308+0000", 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5600000000000005e-07 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.987312+0000", 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0330000000000002e-06 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.987321+0000", 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9700000000000005e-06 2024-09-15T00:09:02.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.987325+0000", 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2259999999999999e-06 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.987366+0000", 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0621999999999998e-05 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.987428+0000", 2024-09-15T00:09:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2157999999999997e-05 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.987661+0000", 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023321299999999999 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:21 3.4 3:20776621:::rbd_header.11291357fc5b:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.988042+0000", 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.106383425, 2024-09-15T00:09:02.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027593099999999998, 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 21 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988042+0000", 2024-09-15T00:09:02.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988042+0000", 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988042+0000", 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7700000000000004e-07 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988046+0000", 2024-09-15T00:09:02.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.681e-06 2024-09-15T00:09:02.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988055+0000", 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1470000000000007e-06 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988060+0000", 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5310000000000003e-06 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988101+0000", 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1205999999999998e-05 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988146+0000", 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5092000000000001e-05 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988318+0000", 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000171597 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.149 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:22 3.4 3:20776621:::rbd_header.11291357fc5b:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.988683+0000", 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.105742527, 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029422200000000002, 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.150 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 22 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988683+0000", 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988683+0000", 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988683+0000", 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3099999999999997e-07 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988688+0000", 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5449999999999997e-06 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988698+0000", 2024-09-15T00:09:02.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9639999999999998e-06 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988705+0000", 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0709999999999997e-06 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988764+0000", 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9191000000000003e-05 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988850+0000", 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6370000000000001e-05 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.988977+0000", 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012635000000000001 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:24 3.4 3:20776621:::rbd_header.11291357fc5b:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.990127+0000", 2024-09-15T00:09:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.104297953, 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00021862800000000001, 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 24 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990127+0000", 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990127+0000", 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990128+0000", 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1399999999999995e-07 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990134+0000", 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3169999999999998e-06 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990150+0000", 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5214999999999999e-05 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990158+0000", 2024-09-15T00:09:02.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4100000000000008e-06 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990216+0000", 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7494000000000002e-05 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990263+0000", 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7250000000000003e-05 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990346+0000", 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3028000000000005e-05 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:25 3.4 3:20776621:::rbd_header.11291357fc5b:head [watch watch cookie 140653333853296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.990779+0000", 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.103646115, 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001152244, 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 25 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990779+0000", 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990779+0000", 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990780+0000", 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9699999999999995e-07 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990791+0000", 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1209000000000001e-05 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990801+0000", 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0565e-05 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990809+0000", 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1389999999999998e-06 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990853+0000", 2024-09-15T00:09:02.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4499999999999997e-05 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990900+0000", 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6394000000000003e-05 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.990995+0000", 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5568000000000004e-05 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:02.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991569+0000", 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057368800000000004 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991857+0000", 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000288671 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991891+0000", 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3068999999999998e-05 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991931+0000", 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0744000000000002e-05 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop_reply(client.4393.0:25 3.4 e23/20 ondisk, result = 0)", 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.991764+0000", 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.102661366, 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000171266, 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991764+0000", 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991764+0000", 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991765+0000", 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8599999999999997e-07 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991769+0000", 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3409999999999996e-06 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991781+0000", 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1766000000000001e-05 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991797+0000", 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5783000000000001e-05 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991848+0000", 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1295999999999998e-05 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991854+0000", 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2759999999999997e-06 2024-09-15T00:09:02.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991935+0000", 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0918e-05 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.169 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:02.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:02.171 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:02.168+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:02.272 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:02.270+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:02.319 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:02.320 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:02.320 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:02.320 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:02.328 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:02.372 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:02.370+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:02.435 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:02.435 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:02.435 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:02.435 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:02.435 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:02.444 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:02.485 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:02.483+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:02.574 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:02.572+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:02.591 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:02.591 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:02.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:02.591 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:02.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.591 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4208.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:08:50.832504+0000", 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.758058551, 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00064811399999999998, 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.832504+0000", 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.832504+0000", 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.832505+0000", 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2499999999999999e-07 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.832512+0000", 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9709999999999998e-06 2024-09-15T00:09:02.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.832526+0000", 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4127999999999999e-05 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.832534+0000", 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8450000000000003e-06 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.832583+0000", 2024-09-15T00:09:02.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9322e-05 2024-09-15T00:09:02.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.832606+0000", 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3428999999999999e-05 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.833115+0000", 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050920000000000002 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:08:50.833152+0000", 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6993999999999999e-05 2024-09-15T00:09:02.596 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4393.0:12 3.4 e23/20 3:20776621:::rbd_header.11291357fc5b:head v 23'1, mlcod=0'0)", 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.976928+0000", 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.61363457799999999, 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070807499999999996, 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.597 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976928+0000", 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976928+0000", 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976929+0000", 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4999999999999999e-07 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976933+0000", 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7910000000000004e-06 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976947+0000", 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3730000000000001e-05 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.976956+0000", 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.422e-06 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977014+0000", 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8306e-05 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977030+0000", 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5948e-05 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977601+0000", 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057147299999999997 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.977636+0000", 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5154999999999998e-05 2024-09-15T00:09:02.601 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4393.0:25 3.4 e23/20 3:20776621:::rbd_header.11291357fc5b:head v 23'2, mlcod=0'0)", 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.991066+0000", 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.59949680400000005, 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00065941299999999997, 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991066+0000", 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991066+0000", 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991067+0000", 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.268e-06 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991088+0000", 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1094000000000002e-05 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991107+0000", 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8346999999999999e-05 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991115+0000", 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7880000000000007e-06 2024-09-15T00:09:02.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991169+0000", 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4243000000000002e-05 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991193+0000", 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4637000000000001e-05 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991688+0000", 2024-09-15T00:09:02.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00049474899999999995 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.991726+0000", 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7286999999999997e-05 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4464.0:12 3.4 e23/20 3:20776621:::rbd_header.11291357fc5b:head v 23'3, mlcod=23'1)", 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:02.062030+0000", 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.52853283299999998, 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00078523299999999998, 2024-09-15T00:09:02.606 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062030+0000", 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062030+0000", 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062032+0000", 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4649999999999999e-06 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062055+0000", 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3847999999999999e-05 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062075+0000", 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9337000000000001e-05 2024-09-15T00:09:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062086+0000", 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0813e-05 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062154+0000", 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8065999999999995e-05 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062179+0000", 2024-09-15T00:09:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5446999999999999e-05 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062778+0000", 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00059899699999999998 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:02.062815+0000", 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7259999999999999e-05 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:02.611 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:02.611 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:02.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:02.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:02.673+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:02.727 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:02.727 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:02.727 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:02.727 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:02.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:02.779 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:02.777+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:02.840 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:02.840 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:02.840 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:02.840 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:02.840 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:02.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:02.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:02.875+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:02.958 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:09:02.993 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.805315+0000", 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.185191164999999, 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081705059999999993, 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:02.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805315+0000", 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805315+0000", 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805317+0000", 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.125e-06 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805326+0000", 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7099999999999996e-06 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:02.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805338+0000", 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1524e-05 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805349+0000", 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1844999999999999e-05 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805414+0000", 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4838000000000005e-05 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805572+0000", 2024-09-15T00:09:02.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015764199999999999 2024-09-15T00:09:02.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:02.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805700+0000", 2024-09-15T00:09:02.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000127868 2024-09-15T00:09:02.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.812761+0000", 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070615469999999996 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.813415+0000", 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00065401200000000004 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.813455+0000", 2024-09-15T00:09:02.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9570000000000002e-05 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.813486+0000", 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0824999999999997e-05 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.867518+0000", 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.122988268, 2024-09-15T00:09:03.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067192019999999996, 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 55 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867518+0000", 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867518+0000", 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867519+0000", 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.44e-07 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867589+0000", 2024-09-15T00:09:03.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9950999999999995e-05 2024-09-15T00:09:03.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867604+0000", 2024-09-15T00:09:03.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5257e-05 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867609+0000", 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9060000000000001e-06 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868263+0000", 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00065423 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868311+0000", 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7880000000000002e-05 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868388+0000", 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6783e-05 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874036+0000", 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005648475 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874171+0000", 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013480999999999999 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874205+0000", 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4143000000000002e-05 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874237+0000", 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2122999999999997e-05 2024-09-15T00:09:03.006 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.867619+0000", 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.122887872, 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066895610000000001, 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 56 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867619+0000", 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867619+0000", 2024-09-15T00:09:03.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867619+0000", 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867621+0000", 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5679999999999998e-06 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867626+0000", 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.211e-06 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867629+0000", 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.852e-06 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868814+0000", 2024-09-15T00:09:03.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011850280000000001 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868864+0000", 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0132e-05 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868959+0000", 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5499000000000002e-05 2024-09-15T00:09:03.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874148+0000", 2024-09-15T00:09:03.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051892659999999997 2024-09-15T00:09:03.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874247+0000", 2024-09-15T00:09:03.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8095000000000002e-05 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874281+0000", 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4493999999999999e-05 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874308+0000", 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7005e-05 2024-09-15T00:09:03.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.911451+0000", 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.079055660000002, 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070195209999999999, 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.014 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911451+0000", 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911451+0000", 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911451+0000", 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6100000000000001e-07 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911455+0000", 2024-09-15T00:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8330000000000004e-06 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911469+0000", 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3687e-05 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911480+0000", 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.132e-05 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913049+0000", 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015688970000000001 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913103+0000", 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4382999999999998e-05 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913172+0000", 2024-09-15T00:09:03.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8921e-05 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914435+0000", 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012632399999999999 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918413+0000", 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039780010000000001 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918447+0000", 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3537000000000001e-05 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918470+0000", 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3241e-05 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.020 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.911493+0000", 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.079013277000001, 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070199570000000003, 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911493+0000", 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911493+0000", 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911494+0000", 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8599999999999998e-07 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911542+0000", 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8383000000000001e-05 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911552+0000", 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6500000000000008e-06 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911557+0000", 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913392+0000", 2024-09-15T00:09:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001834507 2024-09-15T00:09:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913437+0000", 2024-09-15T00:09:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5303999999999999e-05 2024-09-15T00:09:03.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913507+0000", 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0393000000000002e-05 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914855+0000", 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013478940000000001 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918474+0000", 2024-09-15T00:09:03.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036189999999999998 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918496+0000", 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1301000000000002e-05 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918513+0000", 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.751e-05 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.911568+0000", 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.078938822000001, 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006985209, 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 88 2024-09-15T00:09:03.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911568+0000", 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911568+0000", 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911568+0000", 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911572+0000", 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.45e-06 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911578+0000", 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3010000000000003e-06 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911583+0000", 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9400000000000001e-06 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914055+0000", 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002471975 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914101+0000", 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5924000000000003e-05 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914171+0000", 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0736999999999997e-05 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.915063+0000", 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089167000000000001 2024-09-15T00:09:03.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918516+0000", 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034527870000000001 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918537+0000", 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1537999999999999e-05 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918553+0000", 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5376999999999999e-05 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062743+0000", 2024-09-15T00:09:03.033 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.927762929, 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075641670000000001, 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062743+0000", 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062743+0000", 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062744+0000", 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062786+0000", 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2277000000000002e-05 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062797+0000", 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0539e-05 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062802+0000", 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.361e-06 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.064487+0000", 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001684547 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.064532+0000", 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5725e-05 2024-09-15T00:09:03.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.064603+0000", 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0937000000000002e-05 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068092+0000", 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034888440000000001 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070222+0000", 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021295250000000002 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070271+0000", 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9126999999999999e-05 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070308+0000", 2024-09-15T00:09:03.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6712999999999999e-05 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062824+0000", 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.927681937999999, 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075942450000000003, 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062824+0000", 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062824+0000", 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062825+0000", 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0800000000000005e-07 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062829+0000", 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8589999999999998e-06 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062836+0000", 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4019999999999996e-06 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062842+0000", 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000003e-06 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065296+0000", 2024-09-15T00:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00245401 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065340+0000", 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3921999999999997e-05 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065411+0000", 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1361999999999999e-05 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068099+0000", 2024-09-15T00:09:03.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026882680000000002 2024-09-15T00:09:03.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070361+0000", 2024-09-15T00:09:03.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002261894 2024-09-15T00:09:03.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070396+0000", 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.447e-05 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070419+0000", 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2898e-05 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062852+0000", 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.927654410999999, 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076621670000000001, 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 111 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062852+0000", 2024-09-15T00:09:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062852+0000", 2024-09-15T00:09:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062852+0000", 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062856+0000", 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.366e-06 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062862+0000", 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062868+0000", 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2419999999999997e-06 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065656+0000", 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002787702 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065717+0000", 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1558999999999998e-05 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065794+0000", 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6885999999999995e-05 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068511+0000", 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027164170000000001 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070466+0000", 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019549200000000002 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070494+0000", 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8308000000000001e-05 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070514+0000", 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0145000000000001e-05 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062876+0000", 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.927630570000002, 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077296480000000004, 2024-09-15T00:09:03.053 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 112 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062876+0000", 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062876+0000", 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062876+0000", 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062922+0000", 2024-09-15T00:09:03.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5522999999999998e-05 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062935+0000", 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3352e-05 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062941+0000", 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5470000000000004e-06 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066017+0000", 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030763819999999999 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066062+0000", 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4912e-05 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066130+0000", 2024-09-15T00:09:03.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8125000000000003e-05 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068513+0000", 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023831989999999999 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070557+0000", 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020441090000000001 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070586+0000", 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8509000000000001e-05 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070605+0000", 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9534999999999998e-05 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062951+0000", 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.927555193, 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00772794, 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 113 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062951+0000", 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062951+0000", 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062952+0000", 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8800000000000003e-07 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062955+0000", 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062961+0000", 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3890000000000002e-06 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062966+0000", 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7489999999999998e-06 2024-09-15T00:09:03.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066471+0000", 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035052320000000001 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066532+0000", 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0637999999999998e-05 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066602+0000", 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9670000000000002e-05 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068517+0000", 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001914919 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070638+0000", 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021212879999999998 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070661+0000", 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3102999999999999e-05 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070679+0000", 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8131e-05 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.065 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062974+0000", 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.927532131, 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077958259999999996, 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 114 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062974+0000", 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062974+0000", 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062975+0000", 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:09:03.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062978+0000", 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1480000000000002e-06 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062984+0000", 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3219999999999998e-06 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062989+0000", 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9259999999999999e-06 2024-09-15T00:09:03.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066814+0000", 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038244889999999999 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066857+0000", 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3578000000000002e-05 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066936+0000", 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8423000000000005e-05 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068519+0000", 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015831409999999999 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070721+0000", 2024-09-15T00:09:03.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022025130000000001 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070748+0000", 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6957e-05 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070770+0000", 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.19e-05 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.063001+0000", 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.927505663000002, 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078454709999999997, 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.072 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 115 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063001+0000", 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063001+0000", 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063001+0000", 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4700000000000002e-07 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063051+0000", 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9401999999999999e-05 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063058+0000", 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7600000000000002e-06 2024-09-15T00:09:03.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063064+0000", 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067213+0000", 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041489170000000002 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067274+0000", 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0825000000000001e-05 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067409+0000", 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013582399999999999 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.069952+0000", 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025423139999999999 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070806+0000", 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000853859 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070830+0000", 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4204000000000001e-05 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070846+0000", 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6390000000000001e-05 2024-09-15T00:09:03.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.063080+0000", 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.927426696000001, 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078653790000000005, 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 116 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063080+0000", 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063080+0000", 2024-09-15T00:09:03.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063080+0000", 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4700000000000001e-07 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063084+0000", 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4570000000000001e-06 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063090+0000", 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7830000000000001e-06 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063095+0000", 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8250000000000004e-06 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067726+0000", 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046304299999999996 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067772+0000", 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6594999999999997e-05 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067843+0000", 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0188e-05 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.069961+0000", 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002118619 2024-09-15T00:09:03.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070889+0000", 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092833 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070917+0000", 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7174000000000001e-05 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070945+0000", 2024-09-15T00:09:03.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8430999999999999e-05 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.063105+0000", 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.927401795000002, 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079108400000000006, 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.084 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 117 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063105+0000", 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063105+0000", 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063105+0000", 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6800000000000001e-07 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063110+0000", 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.293e-06 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063121+0000", 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1068e-05 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063127+0000", 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0989999999999998e-06 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068237+0000", 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051106720000000001 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068294+0000", 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6354000000000003e-05 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068366+0000", 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2652999999999997e-05 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070616+0000", 2024-09-15T00:09:03.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022500749999999998 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070971+0000", 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035480700000000001 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070997+0000", 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5905999999999999e-05 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.071015+0000", 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8445e-05 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.373113+0000", 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.617393503999999, 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006661167, 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.090 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 139 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373113+0000", 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373113+0000", 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373113+0000", 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6199999999999998e-07 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373120+0000", 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1099999999999997e-06 2024-09-15T00:09:03.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373129+0000", 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0679999999999997e-06 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373137+0000", 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0790000000000001e-06 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.376789+0000", 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036521850000000001 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.376837+0000", 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8538000000000002e-05 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.376901+0000", 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3958e-05 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.377773+0000", 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087204899999999996 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379724+0000", 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001950481 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379755+0000", 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1618000000000001e-05 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379774+0000", 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8618999999999999e-05 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.373146+0000", 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.617360732000002, 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066998939999999996, 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.096 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 140 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373146+0000", 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373146+0000", 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373146+0000", 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000002e-07 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373216+0000", 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9774e-05 2024-09-15T00:09:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373223+0000", 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4939999999999997e-06 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373228+0000", 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9779999999999996e-06 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.377147+0000", 2024-09-15T00:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003918464 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.377192+0000", 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5470999999999999e-05 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.377263+0000", 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1110000000000002e-05 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378345+0000", 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010819359999999999 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379804+0000", 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001458595 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379828+0000", 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4172000000000002e-05 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379846+0000", 2024-09-15T00:09:03.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7626999999999999e-05 2024-09-15T00:09:03.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.102 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.102 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.373237+0000", 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.617269593, 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066716279999999998, 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 141 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373237+0000", 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373237+0000", 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373237+0000", 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8099999999999998e-07 2024-09-15T00:09:03.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373243+0000", 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5060000000000003e-06 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373251+0000", 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.072e-06 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373256+0000", 2024-09-15T00:09:03.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0309999999999998e-06 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378036+0000", 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047805390000000003 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378078+0000", 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1504000000000001e-05 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378145+0000", 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6908000000000004e-05 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378600+0000", 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00045539300000000001 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379866+0000", 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012655539999999999 2024-09-15T00:09:03.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379892+0000", 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6154999999999999e-05 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379908+0000", 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6585000000000001e-05 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.797279+0000", 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.193227469, 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083034489999999992, 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 158 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797279+0000", 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797279+0000", 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797279+0000", 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2000000000000001e-07 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797311+0000", 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1927000000000001e-05 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797316+0000", 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4229999999999996e-06 2024-09-15T00:09:03.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797323+0000", 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.438e-06 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.799837+0000", 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025140570000000001 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.799896+0000", 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8644999999999998e-05 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.799982+0000", 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6878000000000004e-05 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.801206+0000", 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012232549999999999 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805493+0000", 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042870260000000002 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805543+0000", 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9728000000000002e-05 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805582+0000", 2024-09-15T00:09:03.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9851999999999997e-05 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.797328+0000", 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.193177977000001, 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083428370000000005, 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 159 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797328+0000", 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797328+0000", 2024-09-15T00:09:03.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797329+0000", 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7399999999999999e-07 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797331+0000", 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.131e-06 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:03.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797334+0000", 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797337+0000", 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6299999999999998e-06 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.800437+0000", 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031001940000000001 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.800498+0000", 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0965999999999999e-05 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.800606+0000", 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107765 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.801437+0000", 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083154300000000002 2024-09-15T00:09:03.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805618+0000", 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041806769999999998 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805648+0000", 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9674000000000001e-05 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805671+0000", 2024-09-15T00:09:03.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3728999999999999e-05 2024-09-15T00:09:03.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.122 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:03.122 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:03.122 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:03.123 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:03.011+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:03.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:03.124 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:03.121+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:03.225 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:03.222+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:03.254 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:03.267 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:03.268 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:03.268 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:03.268 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:03.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:03.325 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:03.323+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:03.378 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:03.378 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:03.379 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:03.379 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:03.379 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:03.379 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:03.379 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:03.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:03.395 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:03.395 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:03.413 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:03.413 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:03.428 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:03.430 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:03.428+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:03.445 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:03.445 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:03.463 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:03.463 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:03.491 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:03.491 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:03.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:03.528 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:03.526+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:03.628 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:03.626+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:03.643 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:03.643 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:03.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:03.729 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:03.727+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:03.752 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xbb7cc00 2024-09-15T00:09:03.765 INFO:teuthology.orchestra.run.smithi092.stdout:1 trunc from 0x0 to 0xbb7cc00 2024-09-15T00:09:03.779 INFO:teuthology.orchestra.run.smithi092.stdout:2 punch from 0x13f0000 to 0x13faa00, (0xaa00 bytes) 2024-09-15T00:09:03.780 INFO:teuthology.orchestra.run.smithi092.stdout:5 write 0x538b000 thru 0x538e321 (0x3322 bytes) 2024-09-15T00:09:03.780 INFO:teuthology.orchestra.run.smithi092.stdout:6 read 0x9fae800 thru 0x9faffa7 (0x17a8 bytes) 2024-09-15T00:09:03.780 INFO:teuthology.orchestra.run.smithi092.stdout:7 punch from 0x3834400 to 0x383fe00, (0xba00 bytes) 2024-09-15T00:09:03.780 INFO:teuthology.orchestra.run.smithi092.stdout:10 write 0x4b09200 thru 0x4b12dbe (0x9bbf bytes) 2024-09-15T00:09:03.780 INFO:teuthology.orchestra.run.smithi092.stdout:12 read 0x2983c00 thru 0x2988515 (0x4916 bytes) 2024-09-15T00:09:03.780 INFO:teuthology.orchestra.run.smithi092.stdout:13 read 0xb8fc00 thru 0xb93302 (0x3703 bytes) 2024-09-15T00:09:03.787 INFO:teuthology.orchestra.run.smithi092.stdout:16 trunc from 0xbb7cc00 to 0x5cda200 2024-09-15T00:09:03.797 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:03.797 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:03.802 INFO:teuthology.orchestra.run.smithi092.stdout:17 read 0xb22000 thru 0xb25454 (0x3455 bytes) 2024-09-15T00:09:03.804 INFO:teuthology.orchestra.run.smithi092.stdout:20 read 0x1086800 thru 0x108e45f (0x7c60 bytes) 2024-09-15T00:09:03.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:03.818 INFO:teuthology.orchestra.run.smithi092.stdout:21 write 0xab88200 thru 0xab8d24e (0x504f bytes) 2024-09-15T00:09:03.819 INFO:teuthology.orchestra.run.smithi092.stdout:22 write 0x11a6000 thru 0x11ad1d4 (0x71d5 bytes) 2024-09-15T00:09:03.822 INFO:teuthology.orchestra.run.smithi092.stdout:27 read 0x5b4cc00 thru 0x5b58ea0 (0xc2a1 bytes) 2024-09-15T00:09:03.822 INFO:teuthology.orchestra.run.smithi092.stdout:29 punch from 0xa1e8c00 to 0xa1f5200, (0xc600 bytes) 2024-09-15T00:09:03.822 INFO:teuthology.orchestra.run.smithi092.stdout:30 read 0xda2800 thru 0xdaa506 (0x7d07 bytes) 2024-09-15T00:09:03.822 INFO:teuthology.orchestra.run.smithi092.stdout:32 read 0x54f4800 thru 0x5500450 (0xbc51 bytes) 2024-09-15T00:09:03.822 INFO:teuthology.orchestra.run.smithi092.stdout:34 write 0x818600 thru 0x821d1e (0x971f bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:35 write 0x204e800 thru 0x20565bc (0x7dbd bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:37 read 0x8e2b600 thru 0x8e34e1d (0x981e bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:38 write 0x6a24c00 thru 0x6a2bde0 (0x71e1 bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:39 read 0x7242e00 thru 0x724a5ca (0x77cb bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:40 write 0x945ce00 thru 0x9460097 (0x3298 bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:41 read 0x84bd600 thru 0x84c87fd (0xb1fe bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:43 read 0x66e2c00 thru 0x66e9adc (0x6edd bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:44 write 0x6dc2e00 thru 0x6dc9572 (0x6773 bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:46 write 0xc0d600 thru 0xc0fa0c (0x240d bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:48 write 0x64f2000 thru 0x64ff6d7 (0xd6d8 bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:49 read 0x7920c00 thru 0x792b410 (0xa811 bytes) 2024-09-15T00:09:03.823 INFO:teuthology.orchestra.run.smithi092.stdout:50 read 0x22b600 thru 0x22d10c (0x1b0d bytes) 2024-09-15T00:09:03.824 INFO:teuthology.orchestra.run.smithi092.stdout:52 read 0x41bac00 thru 0x41be528 (0x3929 bytes) 2024-09-15T00:09:03.824 INFO:teuthology.orchestra.run.smithi092.stdout:53 trunc from 0xab8d400 to 0x8c29a00 2024-09-15T00:09:03.849 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:03.847+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:03.859 INFO:teuthology.orchestra.run.smithi092.stdout:58 write 0x5af3200 thru 0x5b00a14 (0xd815 bytes) 2024-09-15T00:09:03.859 INFO:teuthology.orchestra.run.smithi092.stdout:59 trunc from 0x8c29a00 to 0x927c800 2024-09-15T00:09:03.871 INFO:teuthology.orchestra.run.smithi092.stdout:61 write 0x7b04a00 thru 0x7b09f62 (0x5563 bytes) 2024-09-15T00:09:03.872 INFO:teuthology.orchestra.run.smithi092.stdout:63 read 0x713fa00 thru 0x714d13b (0xd73c bytes) 2024-09-15T00:09:03.873 INFO:teuthology.orchestra.run.smithi092.stdout:64 read 0x271ec00 thru 0x272dac3 (0xeec4 bytes) 2024-09-15T00:09:03.873 INFO:teuthology.orchestra.run.smithi092.stdout:65 punch from 0x3af1400 to 0x3af5800, (0x4400 bytes) 2024-09-15T00:09:03.873 INFO:teuthology.orchestra.run.smithi092.stdout:68 write 0x9b6200 thru 0x9b79d8 (0x17d9 bytes) 2024-09-15T00:09:03.874 INFO:teuthology.orchestra.run.smithi092.stdout:70 trunc from 0x927c800 to 0x89c1a00 2024-09-15T00:09:03.886 INFO:teuthology.orchestra.run.smithi092.stdout:71 read 0x43ba600 thru 0x43c09ed (0x63ee bytes) 2024-09-15T00:09:03.902 INFO:teuthology.orchestra.run.smithi092.stdout:72 write 0xc9a6000 thru 0xc9b20fa (0xc0fb bytes) 2024-09-15T00:09:03.903 INFO:teuthology.orchestra.run.smithi092.stdout:74 punch from 0x1bb9400 to 0x1bc2200, (0x8e00 bytes) 2024-09-15T00:09:03.904 INFO:teuthology.orchestra.run.smithi092.stdout:78 read 0xbe17200 thru 0xbe271d5 (0xffd6 bytes) 2024-09-15T00:09:03.904 INFO:teuthology.orchestra.run.smithi092.stdout:81 write 0x2c7e000 thru 0x2c86f6e (0x8f6f bytes) 2024-09-15T00:09:03.917 INFO:teuthology.orchestra.run.smithi092.stdout:83 write 0xe841800 thru 0xe844226 (0x2a27 bytes) 2024-09-15T00:09:03.918 INFO:teuthology.orchestra.run.smithi092.stdout:84 read 0x71cfc00 thru 0x71dc449 (0xc84a bytes) 2024-09-15T00:09:03.918 INFO:teuthology.orchestra.run.smithi092.stdout:86 read 0xaf62600 thru 0xaf65f51 (0x3952 bytes) 2024-09-15T00:09:03.918 INFO:teuthology.orchestra.run.smithi092.stdout:88 read 0x8139400 thru 0x81417e4 (0x83e5 bytes) 2024-09-15T00:09:03.918 INFO:teuthology.orchestra.run.smithi092.stdout:90 write 0xb66d000 thru 0xb6731ea (0x61eb bytes) 2024-09-15T00:09:03.918 INFO:teuthology.orchestra.run.smithi092.stdout:93 write 0xa088800 thru 0xa08ff38 (0x7739 bytes) 2024-09-15T00:09:03.919 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:03.919 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:03.922 INFO:teuthology.orchestra.run.smithi092.stdout:94 trunc from 0xe844400 to 0xb5d6400 2024-09-15T00:09:03.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:03.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:03.929+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:03.949 INFO:teuthology.orchestra.run.smithi092.stdout:95 read 0x861d200 thru 0x862ae1e (0xdc1f bytes) 2024-09-15T00:09:03.950 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xd811200 2024-09-15T00:09:03.953 INFO:teuthology.orchestra.run.smithi092.stdout:97 trunc from 0xb5d6400 to 0xd811200 2024-09-15T00:09:03.974 INFO:teuthology.orchestra.run.smithi092.stdout:100 trunc from 0xd811200 to 0x4317800 2024-09-15T00:09:03.994 INFO:teuthology.orchestra.run.smithi092.stdout:102 read 0x342d200 thru 0x34308b5 (0x36b6 bytes) 2024-09-15T00:09:03.994 INFO:teuthology.orchestra.run.smithi092.stdout:103 read 0x2d53a00 thru 0x2d5b7f7 (0x7df8 bytes) 2024-09-15T00:09:04.002 INFO:teuthology.orchestra.run.smithi092.stdout:104 trunc from 0x4317800 to 0x9e58e00 2024-09-15T00:09:04.012 INFO:teuthology.orchestra.run.smithi092.stdout:108 read 0x397ee00 thru 0x3984b56 (0x5d57 bytes) 2024-09-15T00:09:04.014 INFO:teuthology.orchestra.run.smithi092.stdout:109 read 0x6683a00 thru 0x66864fc (0x2afd bytes) 2024-09-15T00:09:04.014 INFO:teuthology.orchestra.run.smithi092.stdout:117 read 0x7754400 thru 0x775518b (0xd8c bytes) 2024-09-15T00:09:04.020 INFO:teuthology.orchestra.run.smithi092.stdout:118 trunc from 0x9e58e00 to 0x4e9ce00 2024-09-15T00:09:04.031 INFO:teuthology.orchestra.run.smithi092.stdout:121 punch from 0x83a400 to 0x83f200, (0x4e00 bytes) 2024-09-15T00:09:04.031 INFO:teuthology.orchestra.run.smithi092.stdout:124 read 0x320cc00 thru 0x321a533 (0xd934 bytes) 2024-09-15T00:09:04.031 INFO:teuthology.orchestra.run.smithi092.stdout:130 write 0x3700800 thru 0x370c914 (0xc115 bytes) 2024-09-15T00:09:04.032 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:04.029+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:04.048 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:04.048 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:04.052 INFO:teuthology.orchestra.run.smithi092.stdout:131 write 0xb496600 thru 0xb499db0 (0x37b1 bytes) 2024-09-15T00:09:04.054 INFO:teuthology.orchestra.run.smithi092.stdout:132 write 0x24ca600 thru 0x24ce374 (0x3d75 bytes) 2024-09-15T00:09:04.054 INFO:teuthology.orchestra.run.smithi092.stdout:133 write 0x466d000 thru 0x467ccca (0xfccb bytes) 2024-09-15T00:09:04.054 INFO:teuthology.orchestra.run.smithi092.stdout:134 punch from 0xb03c800 to 0xb04aa00, (0xe200 bytes) 2024-09-15T00:09:04.054 INFO:teuthology.orchestra.run.smithi092.stdout:135 write 0x3c2e200 thru 0x3c2f75a (0x155b bytes) 2024-09-15T00:09:04.055 INFO:teuthology.orchestra.run.smithi092.stdout:137 punch from 0x8117e00 to 0x811a600, (0x2800 bytes) 2024-09-15T00:09:04.055 INFO:teuthology.orchestra.run.smithi092.stdout:138 read 0xacc9400 thru 0xaccc3cf (0x2fd0 bytes) 2024-09-15T00:09:04.055 INFO:teuthology.orchestra.run.smithi092.stdout:142 read 0xa97bc00 thru 0xa9818a5 (0x5ca6 bytes) 2024-09-15T00:09:04.055 INFO:teuthology.orchestra.run.smithi092.stdout:144 read 0x74d7a00 thru 0x74e014b (0x874c bytes) 2024-09-15T00:09:04.055 INFO:teuthology.orchestra.run.smithi092.stdout:145 punch from 0x440be00 to 0x440ec00, (0x2e00 bytes) 2024-09-15T00:09:04.055 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xde7ea00 2024-09-15T00:09:04.058 INFO:teuthology.orchestra.run.smithi092.stdout:146 trunc from 0xb499e00 to 0xde7ea00 2024-09-15T00:09:04.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:04.070 INFO:teuthology.orchestra.run.smithi092.stdout:147 read 0x94d800 thru 0x94f759 (0x1f5a bytes) 2024-09-15T00:09:04.080 INFO:teuthology.orchestra.run.smithi092.stdout:148 write 0xe78aa00 thru 0xe79705b (0xc65c bytes) 2024-09-15T00:09:04.082 INFO:teuthology.orchestra.run.smithi092.stdout:150 write 0x6528200 thru 0x6534d2b (0xcb2c bytes) 2024-09-15T00:09:04.082 INFO:teuthology.orchestra.run.smithi092.stdout:151 write 0x1f2b800 thru 0x1f33761 (0x7f62 bytes) 2024-09-15T00:09:04.082 INFO:teuthology.orchestra.run.smithi092.stdout:152 write 0x2da7800 thru 0x2daf701 (0x7f02 bytes) 2024-09-15T00:09:04.083 INFO:teuthology.orchestra.run.smithi092.stdout:156 read 0x3ffec00 thru 0x4006da7 (0x81a8 bytes) 2024-09-15T00:09:04.083 INFO:teuthology.orchestra.run.smithi092.stdout:157 punch from 0xfd0800 to 0xfd1c00, (0x1400 bytes) 2024-09-15T00:09:04.093 INFO:teuthology.orchestra.run.smithi092.stdout:158 trunc from 0xe797200 to 0x67bf200 2024-09-15T00:09:04.113 INFO:teuthology.orchestra.run.smithi092.stdout:159 read 0x1082400 thru 0x108cc9f (0xa8a0 bytes) 2024-09-15T00:09:04.114 INFO:teuthology.orchestra.run.smithi092.stdout:161 read 0x729000 thru 0x736407 (0xd408 bytes) 2024-09-15T00:09:04.114 INFO:teuthology.orchestra.run.smithi092.stdout:164 read 0x54a7600 thru 0x54b56bf (0xe0c0 bytes) 2024-09-15T00:09:04.134 INFO:teuthology.orchestra.run.smithi092.stdout:165 write 0xde70200 thru 0xde7cb0d (0xc90e bytes) 2024-09-15T00:09:04.136 INFO:teuthology.orchestra.run.smithi092.stdout:166 read 0x22f4a00 thru 0x22f6bf7 (0x21f8 bytes) 2024-09-15T00:09:04.142 INFO:teuthology.orchestra.run.smithi092.stdout:167 trunc from 0xde7cc00 to 0x9c91c00 2024-09-15T00:09:04.144 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:04.142+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:04.161 INFO:teuthology.orchestra.run.smithi092.stdout:168 read 0x3a29800 thru 0x3a30b5d (0x735e bytes) 2024-09-15T00:09:04.162 INFO:teuthology.orchestra.run.smithi092.stdout:171 read 0xf1f600 thru 0xf236be (0x40bf bytes) 2024-09-15T00:09:04.174 INFO:teuthology.orchestra.run.smithi092.stdout:174 write 0xbbfe600 thru 0xbbfe8a5 (0x2a6 bytes) 2024-09-15T00:09:04.176 INFO:teuthology.orchestra.run.smithi092.stdout:175 read 0x398ec00 thru 0x399499b (0x5d9c bytes) 2024-09-15T00:09:04.176 INFO:teuthology.orchestra.run.smithi092.stdout:176 write 0x1c52000 thru 0x1c5b233 (0x9234 bytes) 2024-09-15T00:09:04.177 INFO:teuthology.orchestra.run.smithi092.stdout:177 read 0xb16fa00 thru 0xb170e23 (0x1424 bytes) 2024-09-15T00:09:04.190 INFO:teuthology.orchestra.run.smithi092.stdout:181 trunc from 0xbbfea00 to 0x83e000 2024-09-15T00:09:04.191 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-15T00:09:04.191 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:09:04.191 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T00:09:04.192 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:09:04.220 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:04.221 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:04.231 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:04.233 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:04.231+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:04.234 INFO:teuthology.orchestra.run.smithi092.stdout:183 write 0x7771c00 thru 0x7776891 (0x4c92 bytes) 2024-09-15T00:09:04.236 INFO:teuthology.orchestra.run.smithi092.stdout:188 write 0x7a4800 thru 0x7b06e6 (0xbee7 bytes) 2024-09-15T00:09:04.236 INFO:teuthology.orchestra.run.smithi092.stdout:189 read 0x196e600 thru 0x1974c09 (0x660a bytes) 2024-09-15T00:09:04.243 INFO:teuthology.orchestra.run.smithi092.stdout:190 trunc from 0x7776a00 to 0x13b3400 2024-09-15T00:09:04.261 INFO:teuthology.orchestra.run.smithi092.stdout:191 read 0xc32400 thru 0xc3941d (0x701e bytes) 2024-09-15T00:09:04.262 INFO:teuthology.orchestra.run.smithi092.stdout:192 read 0x3e4e00 thru 0x3e811c (0x331d bytes) 2024-09-15T00:09:04.262 INFO:teuthology.orchestra.run.smithi092.stdout:195 read 0xcb5800 thru 0xcc1026 (0xb827 bytes) 2024-09-15T00:09:04.264 INFO:teuthology.orchestra.run.smithi092.stdout:197 trunc from 0x13b3400 to 0x22dc00 2024-09-15T00:09:04.293 INFO:teuthology.orchestra.run.smithi092.stdout:198 write 0x1293800 thru 0x12a1f8b (0xe78c bytes) 2024-09-15T00:09:04.294 INFO:teuthology.orchestra.run.smithi092.stdout:199 punch from 0x126ce00 to 0x1276c00, (0x9e00 bytes) 2024-09-15T00:09:04.294 INFO:teuthology.orchestra.run.smithi092.stdout:200 read 0x97f400 thru 0x988dd8 (0x99d9 bytes) 2024-09-15T00:09:04.294 INFO:teuthology.orchestra.run.smithi092.stdout:201 read 0x11d9400 thru 0x11dacde (0x18df bytes) 2024-09-15T00:09:04.294 INFO:teuthology.orchestra.run.smithi092.stdout:202 read 0x929800 thru 0x92d7b0 (0x3fb1 bytes) 2024-09-15T00:09:04.294 INFO:teuthology.orchestra.run.smithi092.stdout:203 read 0x11ea400 thru 0x11f68d0 (0xc4d1 bytes) 2024-09-15T00:09:04.294 INFO:teuthology.orchestra.run.smithi092.stdout:204 clone 1 order 21 su 131072 sc 12 2024-09-15T00:09:04.334 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:04.333+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:04.384 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:04.384 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:04.393 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:04.438 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:04.436+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 24, 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "4f532d31-28c1-4236-a74e-6392960baf2b", 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2024-09-15T00:08:32.007191+0000", 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2024-09-15T00:09:02.951012+0000", 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2024-09-15T00:08:42.794810+0000", 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2024-09-15T00:08:33.890963+0000", 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:09:04.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5806080, 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "nodeep-scrub", 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "noscrub", 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:09:04.502 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 3, 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 6, 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "squid", 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:42.896076+0000", 2024-09-15T00:09:04.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:04.504 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:04.505 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "14", 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:04.506 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:04.507 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:04.508 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:04.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:04.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.508 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:04.508 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:04.508 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:04.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:04.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:04.509 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:04.509 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:04.509 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:04.509 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:04.509 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:04.510 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:04.510 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2024-09-15T00:09:04.510 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2024-09-15T00:09:04.510 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2024-09-15T00:09:04.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.510 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:04.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2024-09-15T00:09:04.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.511 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:04.511 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:04.512 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 6.059999942779541, 2024-09-15T00:09:04.512 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 6.059999942779541, 2024-09-15T00:09:04.512 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.33000001311302185, 2024-09-15T00:09:04.512 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T00:09:04.512 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T00:09:04.512 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 2, 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "rbd", 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:49.173060+0000", 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:04.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:04.514 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 8, 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 8, 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 8, 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 8, 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 8, 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:04.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "18", 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 18, 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:04.516 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:04.517 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:04.518 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:04.519 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "pool_client.0", 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:58.879058+0000", 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:04.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2024-09-15T00:09:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:04.522 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "23", 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 23, 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:04.523 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:04.524 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:04.525 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:04.526 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "56600dc2-b95d-47a7-b2b1-5ddc54c08341", 2024-09-15T00:09:04.527 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:04.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:04.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/1440638768", 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/1440638768", 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6823/1440638768", 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6821/1440638768", 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "ea8f0155-8472-4e87-89fd-7716d05065d0", 2024-09-15T00:09:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.537 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:04.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/3601510193", 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/3601510193", 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/3601510193", 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/3601510193", 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:04.539 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "357d2d59-66f0-47b8-87c8-4acdfe62a07d", 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:04.540 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.541 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:04.542 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:04.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/3222759256", 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/3222759256", 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/3222759256", 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/3222759256", 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:04.545 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "22208f56-d8aa-4a56-a42c-5a990962b860", 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:04.546 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6809", 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6811", 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:04.547 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6813", 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6816", 2024-09-15T00:09:04.548 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6822", 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6823", 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6818", 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:04.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6821", 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6811/675428777", 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6816/675428777", 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6823/675428777", 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6821/675428777", 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:04.551 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "a974629e-9ef0-40a3-94bd-382e0bfa8bb6", 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:04.552 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6808", 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6810", 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6812", 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6814", 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6819", 2024-09-15T00:09:04.555 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6820", 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6815", 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6817", 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6810/3736884138", 2024-09-15T00:09:04.557 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6814/3736884138", 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6820/3736884138", 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6817/3736884138", 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "06b73c2d-466a-4c4e-89fa-31d6eeb1af1d", 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:04.558 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6800", 2024-09-15T00:09:04.559 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6801", 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6802", 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6803", 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6806", 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6807", 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:04.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6804", 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6805", 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6801/2235671634", 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6803/2235671634", 2024-09-15T00:09:04.563 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6807/2235671634", 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6805/2235671634", 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:04.564 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.730107+0000", 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:04.565 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.365297+0000", 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:04.566 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.868500+0000", 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.788781+0000", 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.892143+0000", 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:04.568 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.990037+0000", 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2024-09-15T00:09:04.569 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [ 2024-09-15T00:09:04.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: "snaps": [ 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: "begin": 2, 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: "length": 1 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2024-09-15T00:09:04.571 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2024-09-15T00:09:04.572 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2024-09-15T00:09:04.572 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2024-09-15T00:09:04.572 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:09:04.572 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:09:04.572 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2024-09-15T00:09:04.572 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:04.572 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:04.573 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:04.573 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:04.573 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:04.575 INFO:tasks.thrashosds.thrasher:j is {'epoch': 24, 'fsid': '4f532d31-28c1-4236-a74e-6392960baf2b', 'created': '2024-09-15T00:08:32.007191+0000', 'modified': '2024-09-15T00:09:02.951012+0000', 'last_up_change': '2024-09-15T00:08:42.794810+0000', 'last_in_change': '2024-09-15T00:08:33.890963+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:08:42.896076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:08:49.173060+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:08:58.879058+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '56600dc2-b95d-47a7-b2b1-5ddc54c08341', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 1440638768}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 1440638768}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 1440638768}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 1440638768}]}, 'public_addr': '172.21.15.27:6817/1440638768', 'cluster_addr': '172.21.15.27:6819/1440638768', 'heartbeat_back_addr': '172.21.15.27:6823/1440638768', 'heartbeat_front_addr': '172.21.15.27:6821/1440638768', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ea8f0155-8472-4e87-89fd-7716d05065d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 3601510193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 3601510193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 3601510193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 3601510193}]}, 'public_addr': '172.21.15.27:6809/3601510193', 'cluster_addr': '172.21.15.27:6811/3601510193', 'heartbeat_back_addr': '172.21.15.27:6815/3601510193', 'heartbeat_front_addr': '172.21.15.27:6813/3601510193', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '357d2d59-66f0-47b8-87c8-4acdfe62a07d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 3222759256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 3222759256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 3222759256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 3222759256}]}, 'public_addr': '172.21.15.27:6801/3222759256', 'cluster_addr': '172.21.15.27:6803/3222759256', 'heartbeat_back_addr': '172.21.15.27:6807/3222759256', 'heartbeat_front_addr': '172.21.15.27:6805/3222759256', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '22208f56-d8aa-4a56-a42c-5a990962b860', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6809', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 675428777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6813', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6816', 'nonce': 675428777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 675428777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 675428777}]}, 'public_addr': '172.21.15.79:6811/675428777', 'cluster_addr': '172.21.15.79:6816/675428777', 'heartbeat_back_addr': '172.21.15.79:6823/675428777', 'heartbeat_front_addr': '172.21.15.79:6821/675428777', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a974629e-9ef0-40a3-94bd-382e0bfa8bb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6810', 'nonce': 3736884138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6814', 'nonce': 3736884138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6819', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6820', 'nonce': 3736884138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6815', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 3736884138}]}, 'public_addr': '172.21.15.79:6810/3736884138', 'cluster_addr': '172.21.15.79:6814/3736884138', 'heartbeat_back_addr': '172.21.15.79:6820/3736884138', 'heartbeat_front_addr': '172.21.15.79:6817/3736884138', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '06b73c2d-466a-4c4e-89fa-31d6eeb1af1d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 2235671634}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2235671634}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2235671634}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2235671634}]}, 'public_addr': '172.21.15.79:6801/2235671634', 'cluster_addr': '172.21.15.79:6803/2235671634', 'heartbeat_back_addr': '172.21.15.79:6807/2235671634', 'heartbeat_front_addr': '172.21.15.79:6805/2235671634', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.730107+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.365297+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.868500+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.788781+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.892143+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.990037+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:09:04.575 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:09:04.577 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:04.575+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:04.678 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:04.676+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:04.723 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:04.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:04.778 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:04.776+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:04.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:04.892 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:04.890+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:04.909 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:09:04.909 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:09:04.922 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-15T00:09:04.856298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":180,"num_read_kb":151,"num_write":167,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163688,"kb_used_data":1500,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067352,"statfs":{"total":579820584960,"available":579652968448,"internally_reserved":0,"allocated":1536000,"data_stored":1670766,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":51,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.922344"},"pg_stats":[{"pgid":"3.1f","version":"23'1","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:02.055260+0000","last_change":"2024-09-15T00:09:01.970297+0000","last_active":"2024-09-15T00:09:02.055260+0000","last_peered":"2024-09-15T00:09:02.055260+0000","last_clean":"2024-09-15T00:09:02.055260+0000","last_became_active":"2024-09-15T00:08:59.945793+0000","last_became_peered":"2024-09-15T00:08:59.945793+0000","last_unstale":"2024-09-15T00:09:02.055260+0000","last_undegraded":"2024-09-15T00:09:02.055260+0000","last_fullsized":"2024-09-15T00:09:02.055260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:07:12.728611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075174400000000004,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.978739+0000","last_change":"2024-09-15T00:09:01.978739+0000","last_active":"2024-09-15T00:09:01.978739+0000","last_peered":"2024-09-15T00:09:01.978739+0000","last_clean":"2024-09-15T00:09:01.978739+0000","last_became_active":"2024-09-15T00:08:59.945524+0000","last_became_peered":"2024-09-15T00:08:59.945524+0000","last_unstale":"2024-09-15T00:09:01.978739+0000","last_undegraded":"2024-09-15T00:09:01.978739+0000","last_fullsized":"2024-09-15T00:09:01.978739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:47:49.360632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031880599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.965351+0000","last_change":"2024-09-15T00:09:01.965351+0000","last_active":"2024-09-15T00:09:01.965351+0000","last_peered":"2024-09-15T00:09:01.965351+0000","last_clean":"2024-09-15T00:09:01.965351+0000","last_became_active":"2024-09-15T00:08:59.947317+0000","last_became_peered":"2024-09-15T00:08:59.947317+0000","last_unstale":"2024-09-15T00:09:01.965351+0000","last_undegraded":"2024-09-15T00:09:01.965351+0000","last_fullsized":"2024-09-15T00:09:01.965351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:10:08.093732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001450004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"23'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.967096+0000","last_change":"2024-09-15T00:09:01.964719+0000","last_active":"2024-09-15T00:09:01.967096+0000","last_peered":"2024-09-15T00:09:01.967096+0000","last_clean":"2024-09-15T00:09:01.967096+0000","last_became_active":"2024-09-15T00:08:59.945681+0000","last_became_peered":"2024-09-15T00:08:59.945681+0000","last_unstale":"2024-09-15T00:09:01.967096+0000","last_undegraded":"2024-09-15T00:09:01.967096+0000","last_fullsized":"2024-09-15T00:09:01.967096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:43:46.004953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090320299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.969947+0000","last_change":"2024-09-15T00:09:01.969947+0000","last_active":"2024-09-15T00:09:01.969947+0000","last_peered":"2024-09-15T00:09:01.969947+0000","last_clean":"2024-09-15T00:09:01.969947+0000","last_became_active":"2024-09-15T00:08:59.945617+0000","last_became_peered":"2024-09-15T00:08:59.945617+0000","last_unstale":"2024-09-15T00:09:01.969947+0000","last_undegraded":"2024-09-15T00:09:01.969947+0000","last_fullsized":"2024-09-15T00:09:01.969947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:51.838160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048451499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"23'2","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:02.085148+0000","last_change":"2024-09-15T00:09:01.961084+0000","last_active":"2024-09-15T00:09:02.085148+0000","last_peered":"2024-09-15T00:09:02.085148+0000","last_clean":"2024-09-15T00:09:02.085148+0000","last_became_active":"2024-09-15T00:08:59.946828+0000","last_became_peered":"2024-09-15T00:08:59.946828+0000","last_unstale":"2024-09-15T00:09:02.085148+0000","last_undegraded":"2024-09-15T00:09:02.085148+0000","last_fullsized":"2024-09-15T00:09:02.085148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:35:25.243832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085056300000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.958081+0000","last_change":"2024-09-15T00:09:01.958081+0000","last_active":"2024-09-15T00:09:01.958081+0000","last_peered":"2024-09-15T00:09:01.958081+0000","last_clean":"2024-09-15T00:09:01.958081+0000","last_became_active":"2024-09-15T00:08:59.942052+0000","last_became_peered":"2024-09-15T00:08:59.942052+0000","last_unstale":"2024-09-15T00:09:01.958081+0000","last_undegraded":"2024-09-15T00:09:01.958081+0000","last_fullsized":"2024-09-15T00:09:01.958081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:59:15.739111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055705900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.978690+0000","last_change":"2024-09-15T00:09:01.978690+0000","last_active":"2024-09-15T00:09:01.978690+0000","last_peered":"2024-09-15T00:09:01.978690+0000","last_clean":"2024-09-15T00:09:01.978690+0000","last_became_active":"2024-09-15T00:08:59.941686+0000","last_became_peered":"2024-09-15T00:08:59.941686+0000","last_unstale":"2024-09-15T00:09:01.978690+0000","last_undegraded":"2024-09-15T00:09:01.978690+0000","last_fullsized":"2024-09-15T00:09:01.978690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:52.393694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000308933,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.970454+0000","last_change":"2024-09-15T00:09:01.970454+0000","last_active":"2024-09-15T00:09:01.970454+0000","last_peered":"2024-09-15T00:09:01.970454+0000","last_clean":"2024-09-15T00:09:01.970454+0000","last_became_active":"2024-09-15T00:08:59.946139+0000","last_became_peered":"2024-09-15T00:08:59.946139+0000","last_unstale":"2024-09-15T00:09:01.970454+0000","last_undegraded":"2024-09-15T00:09:01.970454+0000","last_fullsized":"2024-09-15T00:09:01.970454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:59:15.283137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084960000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.965029+0000","last_change":"2024-09-15T00:09:01.965029+0000","last_active":"2024-09-15T00:09:01.965029+0000","last_peered":"2024-09-15T00:09:01.965029+0000","last_clean":"2024-09-15T00:09:01.965029+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:09:01.965029+0000","last_undegraded":"2024-09-15T00:09:01.965029+0000","last_fullsized":"2024-09-15T00:09:01.965029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:41:51.632965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062487,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.953424+0000","last_change":"2024-09-15T00:09:01.953424+0000","last_active":"2024-09-15T00:09:01.953424+0000","last_peered":"2024-09-15T00:09:01.953424+0000","last_clean":"2024-09-15T00:09:01.953424+0000","last_became_active":"2024-09-15T00:08:59.941565+0000","last_became_peered":"2024-09-15T00:08:59.941565+0000","last_unstale":"2024-09-15T00:09:01.953424+0000","last_undegraded":"2024-09-15T00:09:01.953424+0000","last_fullsized":"2024-09-15T00:09:01.953424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:49:06.820716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044338899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.953831+0000","last_change":"2024-09-15T00:09:01.953831+0000","last_active":"2024-09-15T00:09:01.953831+0000","last_peered":"2024-09-15T00:09:01.953831+0000","last_clean":"2024-09-15T00:09:01.953831+0000","last_became_active":"2024-09-15T00:08:59.942776+0000","last_became_peered":"2024-09-15T00:08:59.942776+0000","last_unstale":"2024-09-15T00:09:01.953831+0000","last_undegraded":"2024-09-15T00:09:01.953831+0000","last_fullsized":"2024-09-15T00:09:01.953831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:57:15.169038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079073999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.953151+0000","last_change":"2024-09-15T00:08:52.835701+0000","last_active":"2024-09-15T00:09:01.953151+0000","last_peered":"2024-09-15T00:09:01.953151+0000","last_clean":"2024-09-15T00:09:01.953151+0000","last_became_active":"2024-09-15T00:08:50.969842+0000","last_became_peered":"2024-09-15T00:08:50.969842+0000","last_unstale":"2024-09-15T00:09:01.953151+0000","last_undegraded":"2024-09-15T00:09:01.953151+0000","last_fullsized":"2024-09-15T00:09:01.953151+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:00.926947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.970380+0000","last_change":"2024-09-15T00:09:01.970380+0000","last_active":"2024-09-15T00:09:01.970380+0000","last_peered":"2024-09-15T00:09:01.970380+0000","last_clean":"2024-09-15T00:09:01.970380+0000","last_became_active":"2024-09-15T00:08:59.946388+0000","last_became_peered":"2024-09-15T00:08:59.946388+0000","last_unstale":"2024-09-15T00:09:01.970380+0000","last_undegraded":"2024-09-15T00:09:01.970380+0000","last_fullsized":"2024-09-15T00:09:01.970380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:40:37.487572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083591999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.957644+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:09:01.957644+0000","last_peered":"2024-09-15T00:09:01.957644+0000","last_clean":"2024-09-15T00:09:01.957644+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:09:01.957644+0000","last_undegraded":"2024-09-15T00:09:01.957644+0000","last_fullsized":"2024-09-15T00:09:01.957644+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.979030+0000","last_change":"2024-09-15T00:09:01.979030+0000","last_active":"2024-09-15T00:09:01.979030+0000","last_peered":"2024-09-15T00:09:01.979030+0000","last_clean":"2024-09-15T00:09:01.979030+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:09:01.979030+0000","last_undegraded":"2024-09-15T00:09:01.979030+0000","last_fullsized":"2024-09-15T00:09:01.979030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:34.296172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059204099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.953188+0000","last_change":"2024-09-15T00:08:52.835997+0000","last_active":"2024-09-15T00:09:01.953188+0000","last_peered":"2024-09-15T00:09:01.953188+0000","last_clean":"2024-09-15T00:09:01.953188+0000","last_became_active":"2024-09-15T00:08:51.070763+0000","last_became_peered":"2024-09-15T00:08:51.070763+0000","last_unstale":"2024-09-15T00:09:01.953188+0000","last_undegraded":"2024-09-15T00:09:01.953188+0000","last_fullsized":"2024-09-15T00:09:01.953188+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:58.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"23'3","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:02.090374+0000","last_change":"2024-09-15T00:09:01.958574+0000","last_active":"2024-09-15T00:09:02.090374+0000","last_peered":"2024-09-15T00:09:02.090374+0000","last_clean":"2024-09-15T00:09:02.090374+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:09:02.090374+0000","last_undegraded":"2024-09-15T00:09:02.090374+0000","last_fullsized":"2024-09-15T00:09:02.090374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:54.164006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070311699999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.957647+0000","last_change":"2024-09-15T00:08:52.842966+0000","last_active":"2024-09-15T00:09:01.957647+0000","last_peered":"2024-09-15T00:09:01.957647+0000","last_clean":"2024-09-15T00:09:01.957647+0000","last_became_active":"2024-09-15T00:08:51.070528+0000","last_became_peered":"2024-09-15T00:08:51.070528+0000","last_unstale":"2024-09-15T00:09:01.957647+0000","last_undegraded":"2024-09-15T00:09:01.957647+0000","last_fullsized":"2024-09-15T00:09:01.957647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:52:05.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.965506+0000","last_change":"2024-09-15T00:09:01.965506+0000","last_active":"2024-09-15T00:09:01.965506+0000","last_peered":"2024-09-15T00:09:01.965506+0000","last_clean":"2024-09-15T00:09:01.965506+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:09:01.965506+0000","last_undegraded":"2024-09-15T00:09:01.965506+0000","last_fullsized":"2024-09-15T00:09:01.965506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:04:56.549352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001375294,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.960514+0000","last_change":"2024-09-15T00:09:01.960514+0000","last_active":"2024-09-15T00:09:01.960514+0000","last_peered":"2024-09-15T00:09:01.960514+0000","last_clean":"2024-09-15T00:09:01.960514+0000","last_became_active":"2024-09-15T00:08:59.944404+0000","last_became_peered":"2024-09-15T00:08:59.944404+0000","last_unstale":"2024-09-15T00:09:01.960514+0000","last_undegraded":"2024-09-15T00:09:01.960514+0000","last_fullsized":"2024-09-15T00:09:01.960514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:06.524213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037381900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.963801+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:09:01.963801+0000","last_peered":"2024-09-15T00:09:01.963801+0000","last_clean":"2024-09-15T00:09:01.963801+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:09:01.963801+0000","last_undegraded":"2024-09-15T00:09:01.963801+0000","last_fullsized":"2024-09-15T00:09:01.963801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.958075+0000","last_change":"2024-09-15T00:09:01.958075+0000","last_active":"2024-09-15T00:09:01.958075+0000","last_peered":"2024-09-15T00:09:01.958075+0000","last_clean":"2024-09-15T00:09:01.958075+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:09:01.958075+0000","last_undegraded":"2024-09-15T00:09:01.958075+0000","last_fullsized":"2024-09-15T00:09:01.958075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:15.079277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000567013,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.978322+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:09:01.978322+0000","last_peered":"2024-09-15T00:09:01.978322+0000","last_clean":"2024-09-15T00:09:01.978322+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:09:01.978322+0000","last_undegraded":"2024-09-15T00:09:01.978322+0000","last_fullsized":"2024-09-15T00:09:01.978322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.969727+0000","last_change":"2024-09-15T00:09:01.969727+0000","last_active":"2024-09-15T00:09:01.969727+0000","last_peered":"2024-09-15T00:09:01.969727+0000","last_clean":"2024-09-15T00:09:01.969727+0000","last_became_active":"2024-09-15T00:08:59.942545+0000","last_became_peered":"2024-09-15T00:08:59.942545+0000","last_unstale":"2024-09-15T00:09:01.969727+0000","last_undegraded":"2024-09-15T00:09:01.969727+0000","last_fullsized":"2024-09-15T00:09:01.969727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:54.763547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000355144,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.952810+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:09:01.952810+0000","last_peered":"2024-09-15T00:09:01.952810+0000","last_clean":"2024-09-15T00:09:01.952810+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:09:01.952810+0000","last_undegraded":"2024-09-15T00:09:01.952810+0000","last_fullsized":"2024-09-15T00:09:01.952810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.958272+0000","last_change":"2024-09-15T00:09:01.958272+0000","last_active":"2024-09-15T00:09:01.958272+0000","last_peered":"2024-09-15T00:09:01.958272+0000","last_clean":"2024-09-15T00:09:01.958272+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:09:01.958272+0000","last_undegraded":"2024-09-15T00:09:01.958272+0000","last_fullsized":"2024-09-15T00:09:01.958272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:47:25.721925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056893999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.963822+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:09:01.963822+0000","last_peered":"2024-09-15T00:09:01.963822+0000","last_clean":"2024-09-15T00:09:01.963822+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:09:01.963822+0000","last_undegraded":"2024-09-15T00:09:01.963822+0000","last_fullsized":"2024-09-15T00:09:01.963822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.952818+0000","last_change":"2024-09-15T00:08:44.804415+0000","last_active":"2024-09-15T00:09:01.952818+0000","last_peered":"2024-09-15T00:09:01.952818+0000","last_clean":"2024-09-15T00:09:01.952818+0000","last_became_active":"2024-09-15T00:08:44.804011+0000","last_became_peered":"2024-09-15T00:08:44.804011+0000","last_unstale":"2024-09-15T00:09:01.952818+0000","last_undegraded":"2024-09-15T00:09:01.952818+0000","last_fullsized":"2024-09-15T00:09:01.952818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:12.388733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.978947+0000","last_change":"2024-09-15T00:09:01.978947+0000","last_active":"2024-09-15T00:09:01.978947+0000","last_peered":"2024-09-15T00:09:01.978947+0000","last_clean":"2024-09-15T00:09:01.978947+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:09:01.978947+0000","last_undegraded":"2024-09-15T00:09:01.978947+0000","last_fullsized":"2024-09-15T00:09:01.978947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:34.194115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051427500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.960508+0000","last_change":"2024-09-15T00:09:01.960508+0000","last_active":"2024-09-15T00:09:01.960508+0000","last_peered":"2024-09-15T00:09:01.960508+0000","last_clean":"2024-09-15T00:09:01.960508+0000","last_became_active":"2024-09-15T00:08:59.945869+0000","last_became_peered":"2024-09-15T00:08:59.945869+0000","last_unstale":"2024-09-15T00:09:01.960508+0000","last_undegraded":"2024-09-15T00:09:01.960508+0000","last_fullsized":"2024-09-15T00:09:01.960508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:10:17.924412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031568600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.953327+0000","last_change":"2024-09-15T00:09:01.953327+0000","last_active":"2024-09-15T00:09:01.953327+0000","last_peered":"2024-09-15T00:09:01.953327+0000","last_clean":"2024-09-15T00:09:01.953327+0000","last_became_active":"2024-09-15T00:08:59.942796+0000","last_became_peered":"2024-09-15T00:08:59.942796+0000","last_unstale":"2024-09-15T00:09:01.953327+0000","last_undegraded":"2024-09-15T00:09:01.953327+0000","last_fullsized":"2024-09-15T00:09:01.953327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:06:16.865247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035268300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.958373+0000","last_change":"2024-09-15T00:09:01.958373+0000","last_active":"2024-09-15T00:09:01.958373+0000","last_peered":"2024-09-15T00:09:01.958373+0000","last_clean":"2024-09-15T00:09:01.958373+0000","last_became_active":"2024-09-15T00:08:59.941757+0000","last_became_peered":"2024-09-15T00:08:59.941757+0000","last_unstale":"2024-09-15T00:09:01.958373+0000","last_undegraded":"2024-09-15T00:09:01.958373+0000","last_fullsized":"2024-09-15T00:09:01.958373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:57:19.523158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071565300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"23'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.981613+0000","last_change":"2024-09-15T00:09:01.964756+0000","last_active":"2024-09-15T00:09:01.981613+0000","last_peered":"2024-09-15T00:09:01.981613+0000","last_clean":"2024-09-15T00:09:01.981613+0000","last_became_active":"2024-09-15T00:08:59.946686+0000","last_became_peered":"2024-09-15T00:08:59.946686+0000","last_unstale":"2024-09-15T00:09:01.981613+0000","last_undegraded":"2024-09-15T00:09:01.981613+0000","last_fullsized":"2024-09-15T00:09:01.981613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:34:30.157508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092954400000000003,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.953837+0000","last_change":"2024-09-15T00:09:01.953837+0000","last_active":"2024-09-15T00:09:01.953837+0000","last_peered":"2024-09-15T00:09:01.953837+0000","last_clean":"2024-09-15T00:09:01.953837+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:09:01.953837+0000","last_undegraded":"2024-09-15T00:09:01.953837+0000","last_fullsized":"2024-09-15T00:09:01.953837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:02:54.607251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078421999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.965099+0000","last_change":"2024-09-15T00:09:01.965099+0000","last_active":"2024-09-15T00:09:01.965099+0000","last_peered":"2024-09-15T00:09:01.965099+0000","last_clean":"2024-09-15T00:09:01.965099+0000","last_became_active":"2024-09-15T00:08:59.945750+0000","last_became_peered":"2024-09-15T00:08:59.945750+0000","last_unstale":"2024-09-15T00:09:01.965099+0000","last_undegraded":"2024-09-15T00:09:01.965099+0000","last_fullsized":"2024-09-15T00:09:01.965099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:26.899647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010923459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.965290+0000","last_change":"2024-09-15T00:09:01.965290+0000","last_active":"2024-09-15T00:09:01.965290+0000","last_peered":"2024-09-15T00:09:01.965290+0000","last_clean":"2024-09-15T00:09:01.965290+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:09:01.965290+0000","last_undegraded":"2024-09-15T00:09:01.965290+0000","last_fullsized":"2024-09-15T00:09:01.965290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:11:34.186864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001242525,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.964202+0000","last_change":"2024-09-15T00:09:01.964202+0000","last_active":"2024-09-15T00:09:01.964202+0000","last_peered":"2024-09-15T00:09:01.964202+0000","last_clean":"2024-09-15T00:09:01.964202+0000","last_became_active":"2024-09-15T00:08:59.945540+0000","last_became_peered":"2024-09-15T00:08:59.945540+0000","last_unstale":"2024-09-15T00:09:01.964202+0000","last_undegraded":"2024-09-15T00:09:01.964202+0000","last_fullsized":"2024-09-15T00:09:01.964202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:56:28.220270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041955600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.953573+0000","last_change":"2024-09-15T00:09:01.953573+0000","last_active":"2024-09-15T00:09:01.953573+0000","last_peered":"2024-09-15T00:09:01.953573+0000","last_clean":"2024-09-15T00:09:01.953573+0000","last_became_active":"2024-09-15T00:08:59.942379+0000","last_became_peered":"2024-09-15T00:08:59.942379+0000","last_unstale":"2024-09-15T00:09:01.953573+0000","last_undegraded":"2024-09-15T00:09:01.953573+0000","last_fullsized":"2024-09-15T00:09:01.953573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:28:09.255646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067808099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.969823+0000","last_change":"2024-09-15T00:09:01.969823+0000","last_active":"2024-09-15T00:09:01.969823+0000","last_peered":"2024-09-15T00:09:01.969823+0000","last_clean":"2024-09-15T00:09:01.969823+0000","last_became_active":"2024-09-15T00:08:59.946308+0000","last_became_peered":"2024-09-15T00:08:59.946308+0000","last_unstale":"2024-09-15T00:09:01.969823+0000","last_undegraded":"2024-09-15T00:09:01.969823+0000","last_fullsized":"2024-09-15T00:09:01.969823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:01:26.925436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041010299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T00:09:01.953644+0000","last_change":"2024-09-15T00:09:01.953644+0000","last_active":"2024-09-15T00:09:01.953644+0000","last_peered":"2024-09-15T00:09:01.953644+0000","last_clean":"2024-09-15T00:09:01.953644+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:09:01.953644+0000","last_undegraded":"2024-09-15T00:09:01.953644+0000","last_fullsized":"2024-09-15T00:09:01.953644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:22:39.828787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061215200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":51,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640263,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640263,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640264,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":672918,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640263,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":82534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640263,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640264,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":672948,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:09:04.924 INFO:tasks.thrashosds.thrasher:Setting 3.1e to [3, 4, 0, 1] 2024-09-15T00:09:04.924 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1e', '3', '4', '0', '1'] 2024-09-15T00:09:04.924 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1e 3 4 0 1 2024-09-15T00:09:04.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:09:05.004 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:05.004 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:05.004 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:05.001+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:05.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:05.080 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:05.079+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:05.170 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:05.170 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:05.179 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:05.181 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:05.179+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:05.242 INFO:teuthology.orchestra.run.smithi092.stdout:207 write 0xe9bfa00 thru 0xe9c3613 (0x3c14 bytes) 2024-09-15T00:09:05.244 INFO:teuthology.orchestra.run.smithi092.stdout:208 read 0x4f66000 thru 0x4f697b8 (0x37b9 bytes) 2024-09-15T00:09:05.244 INFO:teuthology.orchestra.run.smithi092.stdout:209 write 0x5424400 thru 0x5433e2e (0xfa2f bytes) 2024-09-15T00:09:05.244 INFO:teuthology.orchestra.run.smithi092.stdout:210 write 0x5eece00 thru 0x5ef6812 (0x9a13 bytes) 2024-09-15T00:09:05.244 INFO:teuthology.orchestra.run.smithi092.stdout:212 punch from 0x9dd2200 to 0x9ddaa00, (0x8800 bytes) 2024-09-15T00:09:05.244 INFO:teuthology.orchestra.run.smithi092.stdout:213 punch from 0x190be00 to 0x1917200, (0xb400 bytes) 2024-09-15T00:09:05.244 INFO:teuthology.orchestra.run.smithi092.stdout:214 write 0xaa15a00 thru 0xaa24990 (0xef91 bytes) 2024-09-15T00:09:05.245 INFO:teuthology.orchestra.run.smithi092.stdout:215 write 0x5420200 thru 0x542933d (0x913e bytes) 2024-09-15T00:09:05.252 INFO:teuthology.orchestra.run.smithi092.stdout:219 write 0xbed2200 thru 0xbedd2c8 (0xb0c9 bytes) 2024-09-15T00:09:05.252 INFO:teuthology.orchestra.run.smithi092.stdout:222 punch from 0xa95c600 to 0xa962c00, (0x6600 bytes) 2024-09-15T00:09:05.255 INFO:teuthology.orchestra.run.smithi092.stdout:224 trunc from 0xe9c3800 to 0xc3e2200 2024-09-15T00:09:05.274 INFO:teuthology.orchestra.run.smithi092.stdout:225 read 0x26ffc00 thru 0x2707b47 (0x7f48 bytes) 2024-09-15T00:09:05.275 INFO:teuthology.orchestra.run.smithi092.stdout:227 read 0xb6bca00 thru 0xb6c92c2 (0xc8c3 bytes) 2024-09-15T00:09:05.277 INFO:teuthology.orchestra.run.smithi092.stdout:228 read 0x5c94200 thru 0x5c9602d (0x1e2e bytes) 2024-09-15T00:09:05.277 INFO:teuthology.orchestra.run.smithi092.stdout:229 punch from 0x27c8c00 to 0x27d5c00, (0xd000 bytes) 2024-09-15T00:09:05.277 INFO:teuthology.orchestra.run.smithi092.stdout:230 write 0x7efa200 thru 0x7efdd22 (0x3b23 bytes) 2024-09-15T00:09:05.277 INFO:teuthology.orchestra.run.smithi092.stdout:231 read 0x8f04800 thru 0x8f10d0b (0xc50c bytes) 2024-09-15T00:09:05.277 INFO:teuthology.orchestra.run.smithi092.stdout:233 write 0xb4b7600 thru 0xb4c286a (0xb26b bytes) 2024-09-15T00:09:05.277 INFO:teuthology.orchestra.run.smithi092.stdout:234 trunc from 0xc3e2200 to 0xc73f600 2024-09-15T00:09:05.281 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:05.280+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:05.293 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:05.293 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:05.295 INFO:teuthology.orchestra.run.smithi092.stdout:235 write 0xe24a00 thru 0xe292ce (0x48cf bytes) 2024-09-15T00:09:05.296 INFO:teuthology.orchestra.run.smithi092.stdout:236 read 0xa9c7600 thru 0xa9ce160 (0x6b61 bytes) 2024-09-15T00:09:05.297 INFO:teuthology.orchestra.run.smithi092.stdout:237 read 0x3e0fc00 thru 0x3e1d8e8 (0xdce9 bytes) 2024-09-15T00:09:05.297 INFO:teuthology.orchestra.run.smithi092.stdout:238 read 0xa299c00 thru 0xa2a24f7 (0x88f8 bytes) 2024-09-15T00:09:05.297 INFO:teuthology.orchestra.run.smithi092.stdout:239 write 0xaad4600 thru 0xaadb9fc (0x73fd bytes) 2024-09-15T00:09:05.298 INFO:teuthology.orchestra.run.smithi092.stdout:241 read 0x9e1f600 thru 0x9e2194a (0x234b bytes) 2024-09-15T00:09:05.299 INFO:teuthology.orchestra.run.smithi092.stdout:242 write 0xab03e00 thru 0xab11f53 (0xe154 bytes) 2024-09-15T00:09:05.299 INFO:teuthology.orchestra.run.smithi092.stdout:244 punch from 0x2c54c00 to 0x2c60800, (0xbc00 bytes) 2024-09-15T00:09:05.299 INFO:teuthology.orchestra.run.smithi092.stdout:246 read 0x448ec00 thru 0x449be92 (0xd293 bytes) 2024-09-15T00:09:05.300 INFO:teuthology.orchestra.run.smithi092.stdout:247 write 0xb167400 thru 0xb174e90 (0xda91 bytes) 2024-09-15T00:09:05.300 INFO:teuthology.orchestra.run.smithi092.stdout:249 punch from 0x70de400 to 0x70e1200, (0x2e00 bytes) 2024-09-15T00:09:05.300 INFO:teuthology.orchestra.run.smithi092.stdout:251 read 0xb925c00 thru 0xb92fbee (0x9fef bytes) 2024-09-15T00:09:05.300 INFO:teuthology.orchestra.run.smithi092.stdout:252 read 0x6775200 thru 0x677dc7c (0x8a7d bytes) 2024-09-15T00:09:05.303 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:05.314 INFO:teuthology.orchestra.run.smithi092.stdout:254 trunc from 0xc73f600 to 0xb00a00 2024-09-15T00:09:05.364 INFO:teuthology.orchestra.run.smithi092.stdout:255 trunc from 0xb00a00 to 0x9c8b200 2024-09-15T00:09:05.373 INFO:teuthology.orchestra.run.smithi092.stdout:256 read 0x869ee00 thru 0x86a14a8 (0x26a9 bytes) 2024-09-15T00:09:05.377 INFO:teuthology.orchestra.run.smithi092.stdout:259 trunc from 0x9c8b200 to 0x6fe5000 2024-09-15T00:09:05.382 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:05.380+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:05.387 INFO:teuthology.orchestra.run.smithi092.stdout:261 punch from 0x17de600 to 0x17e8400, (0x9e00 bytes) 2024-09-15T00:09:05.389 INFO:teuthology.orchestra.run.smithi092.stdout:262 punch from 0xc77800 to 0xc86200, (0xea00 bytes) 2024-09-15T00:09:05.389 INFO:teuthology.orchestra.run.smithi092.stdout:263 punch from 0x47a9a00 to 0x47b5800, (0xbe00 bytes) 2024-09-15T00:09:05.390 INFO:teuthology.orchestra.run.smithi092.stdout:265 trunc from 0x6fe5000 to 0x826e400 2024-09-15T00:09:05.407 INFO:teuthology.orchestra.run.smithi092.stdout:267 trunc from 0x826e400 to 0x469d600 2024-09-15T00:09:05.418 INFO:teuthology.orchestra.run.smithi092.stdout:268 punch from 0x190e600 to 0x191b000, (0xca00 bytes) 2024-09-15T00:09:05.419 INFO:teuthology.orchestra.run.smithi092.stdout:269 punch from 0x972a00 to 0x976400, (0x3a00 bytes) 2024-09-15T00:09:05.426 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:05.426 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:05.435 INFO:teuthology.orchestra.run.smithi092.stdout:270 write 0x877d000 thru 0x877dedc (0xedd bytes) 2024-09-15T00:09:05.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:05.442 INFO:teuthology.orchestra.run.smithi092.stdout:272 read 0x275d200 thru 0x2768d69 (0xbb6a bytes) 2024-09-15T00:09:05.442 INFO:teuthology.orchestra.run.smithi092.stdout:273 read 0x5397a00 thru 0x539c817 (0x4e18 bytes) 2024-09-15T00:09:05.443 INFO:teuthology.orchestra.run.smithi092.stdout:274 punch from 0x41e4200 to 0x41f3a00, (0xf800 bytes) 2024-09-15T00:09:05.460 INFO:teuthology.orchestra.run.smithi092.stdout:275 write 0xade2400 thru 0xade7ecf (0x5ad0 bytes) 2024-09-15T00:09:05.462 INFO:teuthology.orchestra.run.smithi092.stdout:276 trunc from 0xade8000 to 0x93d7c00 2024-09-15T00:09:05.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:05.481 INFO:teuthology.orchestra.run.smithi092.stdout:277 read 0x649de00 thru 0x64ab20b (0xd40c bytes) 2024-09-15T00:09:05.482 INFO:teuthology.orchestra.run.smithi092.stdout:279 punch from 0x14a1400 to 0x14a2c00, (0x1800 bytes) 2024-09-15T00:09:05.485 INFO:teuthology.orchestra.run.smithi092.stdout:281 trunc from 0x93d7c00 to 0x78c7600 2024-09-15T00:09:05.496 INFO:teuthology.orchestra.run.smithi092.stdout:283 read 0x33c1a00 thru 0x33c7fa2 (0x65a3 bytes) 2024-09-15T00:09:05.497 INFO:teuthology.orchestra.run.smithi092.stdout:284 punch from 0x5deea00 to 0x5df8400, (0x9a00 bytes) 2024-09-15T00:09:05.497 INFO:teuthology.orchestra.run.smithi092.stdout:286 write 0x1b61000 thru 0x1b64988 (0x3989 bytes) 2024-09-15T00:09:05.497 INFO:teuthology.orchestra.run.smithi092.stdout:287 punch from 0xa50a00 to 0xa5c400, (0xba00 bytes) 2024-09-15T00:09:05.504 INFO:teuthology.orchestra.run.smithi092.stdout:290 trunc from 0x78c7600 to 0x25dcc00 2024-09-15T00:09:05.519 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:05.517+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:05.520 INFO:teuthology.orchestra.run.smithi092.stdout:295 punch from 0x184a000 to 0x184cc00, (0x2c00 bytes) 2024-09-15T00:09:05.538 INFO:teuthology.orchestra.run.smithi092.stdout:298 write 0x8dad000 thru 0x8db49e0 (0x79e1 bytes) 2024-09-15T00:09:05.540 INFO:teuthology.orchestra.run.smithi092.stdout:skipping zero length punch hole 2024-09-15T00:09:05.540 INFO:teuthology.orchestra.run.smithi092.stdout:302 trunc from 0x8db4a00 to 0x8198600 2024-09-15T00:09:05.553 INFO:teuthology.orchestra.run.smithi092.stdout:303 trunc from 0x8198600 to 0x7d48c00 2024-09-15T00:09:05.565 INFO:teuthology.orchestra.run.smithi092.stdout:304 punch from 0x2338c00 to 0x2341600, (0x8a00 bytes) 2024-09-15T00:09:05.567 INFO:teuthology.orchestra.run.smithi092.stdout:305 punch from 0x6b3fa00 to 0x6b4a600, (0xac00 bytes) 2024-09-15T00:09:05.567 INFO:teuthology.orchestra.run.smithi092.stdout:307 read 0x41dec00 thru 0x41e239d (0x379e bytes) 2024-09-15T00:09:05.567 INFO:teuthology.orchestra.run.smithi092.stdout:310 read 0x3bb200 thru 0x3c06f7 (0x54f8 bytes) 2024-09-15T00:09:05.567 INFO:teuthology.orchestra.run.smithi092.stdout:311 write 0x3f2e00 thru 0x3fd2c6 (0xa4c7 bytes) 2024-09-15T00:09:05.575 INFO:teuthology.orchestra.run.smithi092.stdout:314 trunc from 0x7d48c00 to 0x22ed000 2024-09-15T00:09:05.583 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:05.582+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:05.593 INFO:teuthology.orchestra.run.smithi092.stdout:316 read 0x898c00 thru 0x8a5125 (0xc526 bytes) 2024-09-15T00:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:05.595 INFO:teuthology.orchestra.run.smithi092.stdout:320 read 0xf8a200 thru 0xf8b2da (0x10db bytes) 2024-09-15T00:09:05.597 INFO:teuthology.orchestra.run.smithi092.stdout:321 trunc from 0x22ed000 to 0x4464c00 2024-09-15T00:09:05.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:05.605 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:05.605 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:05.606 INFO:teuthology.orchestra.run.smithi092.stdout:322 read 0x2f27600 thru 0x2f2797d (0x37e bytes) 2024-09-15T00:09:05.613 INFO:teuthology.orchestra.run.smithi092.stdout:323 trunc from 0x4464c00 to 0x93a7600 2024-09-15T00:09:05.623 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:05.624 INFO:teuthology.orchestra.run.smithi092.stdout:325 write 0x916f600 thru 0x917d5d0 (0xdfd1 bytes) 2024-09-15T00:09:05.630 INFO:teuthology.orchestra.run.smithi092.stdout:326 trunc from 0x93a7600 to 0xda29200 2024-09-15T00:09:05.641 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:05.641 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:05.646 INFO:teuthology.orchestra.run.smithi092.stdout:328 punch from 0xbc55a00 to 0xbc62c00, (0xd200 bytes) 2024-09-15T00:09:05.647 INFO:teuthology.orchestra.run.smithi092.stdout:329 clone 2 order 19 su 16384 sc 5 2024-09-15T00:09:05.660 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:05.660 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:05.677 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:05.688 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:05.687+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:05.693 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:05.694 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.805315+0000", 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.942672256000002, 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081705059999999993, 2024-09-15T00:09:05.751 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805315+0000", 2024-09-15T00:09:05.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805315+0000", 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805317+0000", 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.125e-06 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805326+0000", 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7099999999999996e-06 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805338+0000", 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1524e-05 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805349+0000", 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1844999999999999e-05 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805414+0000", 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4838000000000005e-05 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805572+0000", 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015764199999999999 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.805700+0000", 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000127868 2024-09-15T00:09:05.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.812761+0000", 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070615469999999996 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.813415+0000", 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00065401200000000004 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.813455+0000", 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9570000000000002e-05 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.813486+0000", 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0824999999999997e-05 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.867518+0000", 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.880469358999999, 2024-09-15T00:09:05.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067192019999999996, 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 55 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867518+0000", 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867518+0000", 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867519+0000", 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.44e-07 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867589+0000", 2024-09-15T00:09:05.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9950999999999995e-05 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867604+0000", 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5257e-05 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867609+0000", 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9060000000000001e-06 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868263+0000", 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00065423 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868311+0000", 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7880000000000002e-05 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868388+0000", 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6783e-05 2024-09-15T00:09:05.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874036+0000", 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005648475 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874171+0000", 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013480999999999999 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874205+0000", 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4143000000000002e-05 2024-09-15T00:09:05.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874237+0000", 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2122999999999997e-05 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.867619+0000", 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.880368962999999, 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066895610000000001, 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.763 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 56 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867619+0000", 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867619+0000", 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867619+0000", 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867621+0000", 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5679999999999998e-06 2024-09-15T00:09:05.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867626+0000", 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.211e-06 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.867629+0000", 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.852e-06 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868814+0000", 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011850280000000001 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868864+0000", 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0132e-05 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.868959+0000", 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5499000000000002e-05 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874148+0000", 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051892659999999997 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874247+0000", 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8095000000000002e-05 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874281+0000", 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4493999999999999e-05 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.874308+0000", 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7005e-05 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:86 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.911451+0000", 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.836536751000001, 2024-09-15T00:09:05.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070195209999999999, 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911451+0000", 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911451+0000", 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911451+0000", 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6100000000000001e-07 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911455+0000", 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8330000000000004e-06 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911469+0000", 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3687e-05 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911480+0000", 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.132e-05 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913049+0000", 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015688970000000001 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913103+0000", 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4382999999999998e-05 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913172+0000", 2024-09-15T00:09:05.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8921e-05 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914435+0000", 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012632399999999999 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918413+0000", 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039780010000000001 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918447+0000", 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3537000000000001e-05 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918470+0000", 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3241e-05 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.775 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.911493+0000", 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.836494368, 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070199570000000003, 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911493+0000", 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911493+0000", 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911494+0000", 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8599999999999998e-07 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911542+0000", 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8383000000000001e-05 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911552+0000", 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6500000000000008e-06 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911557+0000", 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913392+0000", 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001834507 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913437+0000", 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5303999999999999e-05 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.913507+0000", 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0393000000000002e-05 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914855+0000", 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013478940000000001 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918474+0000", 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036189999999999998 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918496+0000", 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1301000000000002e-05 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918513+0000", 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.751e-05 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:44.911568+0000", 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.836419913, 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006985209, 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 88 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911568+0000", 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911568+0000", 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911568+0000", 2024-09-15T00:09:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911572+0000", 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.45e-06 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911578+0000", 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3010000000000003e-06 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.911583+0000", 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9400000000000001e-06 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914055+0000", 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002471975 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914101+0000", 2024-09-15T00:09:05.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5924000000000003e-05 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.914171+0000", 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0736999999999997e-05 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.915063+0000", 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089167000000000001 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918516+0000", 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034527870000000001 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918537+0000", 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1537999999999999e-05 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:44.918553+0000", 2024-09-15T00:09:05.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5376999999999999e-05 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062743+0000", 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.685244019999999, 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075641670000000001, 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062743+0000", 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062743+0000", 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062744+0000", 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7199999999999999e-07 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062786+0000", 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2277000000000002e-05 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062797+0000", 2024-09-15T00:09:05.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0539e-05 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062802+0000", 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.361e-06 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.064487+0000", 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001684547 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.064532+0000", 2024-09-15T00:09:05.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5725e-05 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.064603+0000", 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0937000000000002e-05 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068092+0000", 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034888440000000001 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070222+0000", 2024-09-15T00:09:05.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021295250000000002 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070271+0000", 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9126999999999999e-05 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070308+0000", 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6712999999999999e-05 2024-09-15T00:09:05.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062824+0000", 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.685163029000002, 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075942450000000003, 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.794 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062824+0000", 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062824+0000", 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062825+0000", 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0800000000000005e-07 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062829+0000", 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8589999999999998e-06 2024-09-15T00:09:05.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062836+0000", 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4019999999999996e-06 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062842+0000", 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000003e-06 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065296+0000", 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00245401 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065340+0000", 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3921999999999997e-05 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065411+0000", 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1361999999999999e-05 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068099+0000", 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026882680000000002 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070361+0000", 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002261894 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070396+0000", 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.447e-05 2024-09-15T00:09:05.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070419+0000", 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2898e-05 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062852+0000", 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.685135502000001, 2024-09-15T00:09:05.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076621670000000001, 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 111 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062852+0000", 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062852+0000", 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062852+0000", 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062856+0000", 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.366e-06 2024-09-15T00:09:05.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062862+0000", 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062868+0000", 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2419999999999997e-06 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065656+0000", 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002787702 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065717+0000", 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1558999999999998e-05 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.065794+0000", 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6885999999999995e-05 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068511+0000", 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027164170000000001 2024-09-15T00:09:05.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070466+0000", 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019549200000000002 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070494+0000", 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8308000000000001e-05 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070514+0000", 2024-09-15T00:09:05.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0145000000000001e-05 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062876+0000", 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.685111661000001, 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077296480000000004, 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 112 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062876+0000", 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062876+0000", 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062876+0000", 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062922+0000", 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5522999999999998e-05 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062935+0000", 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3352e-05 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062941+0000", 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5470000000000004e-06 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066017+0000", 2024-09-15T00:09:05.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030763819999999999 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066062+0000", 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4912e-05 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066130+0000", 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8125000000000003e-05 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068513+0000", 2024-09-15T00:09:05.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023831989999999999 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070557+0000", 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020441090000000001 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070586+0000", 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8509000000000001e-05 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070605+0000", 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9534999999999998e-05 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062951+0000", 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.685036283999999, 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00772794, 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 113 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062951+0000", 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062951+0000", 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062952+0000", 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8800000000000003e-07 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062955+0000", 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062961+0000", 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3890000000000002e-06 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062966+0000", 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7489999999999998e-06 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066471+0000", 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035052320000000001 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066532+0000", 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0637999999999998e-05 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066602+0000", 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9670000000000002e-05 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068517+0000", 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001914919 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070638+0000", 2024-09-15T00:09:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021212879999999998 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070661+0000", 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3102999999999999e-05 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070679+0000", 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8131e-05 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.062974+0000", 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.685013221999998, 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077958259999999996, 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 114 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062974+0000", 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062974+0000", 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062975+0000", 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062978+0000", 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1480000000000002e-06 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062984+0000", 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3219999999999998e-06 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.062989+0000", 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9259999999999999e-06 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066814+0000", 2024-09-15T00:09:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038244889999999999 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066857+0000", 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3578000000000002e-05 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.066936+0000", 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8423000000000005e-05 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068519+0000", 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015831409999999999 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070721+0000", 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022025130000000001 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070748+0000", 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6957e-05 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070770+0000", 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.19e-05 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.063001+0000", 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.684986754000001, 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078454709999999997, 2024-09-15T00:09:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 115 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063001+0000", 2024-09-15T00:09:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063001+0000", 2024-09-15T00:09:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063001+0000", 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4700000000000002e-07 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063051+0000", 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9401999999999999e-05 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063058+0000", 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7600000000000002e-06 2024-09-15T00:09:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063064+0000", 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067213+0000", 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041489170000000002 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067274+0000", 2024-09-15T00:09:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0825000000000001e-05 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067409+0000", 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013582399999999999 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.069952+0000", 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025423139999999999 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070806+0000", 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000853859 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070830+0000", 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4204000000000001e-05 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070846+0000", 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6390000000000001e-05 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.063080+0000", 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.684907787, 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078653790000000005, 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 116 2024-09-15T00:09:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063080+0000", 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063080+0000", 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063080+0000", 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4700000000000001e-07 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063084+0000", 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4570000000000001e-06 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063090+0000", 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7830000000000001e-06 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063095+0000", 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8250000000000004e-06 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067726+0000", 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046304299999999996 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067772+0000", 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6594999999999997e-05 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.067843+0000", 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0188e-05 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.069961+0000", 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002118619 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070889+0000", 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092833 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070917+0000", 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7174000000000001e-05 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070945+0000", 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8430999999999999e-05 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.063105+0000", 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.684882886, 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079108400000000006, 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 117 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063105+0000", 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063105+0000", 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063105+0000", 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6800000000000001e-07 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063110+0000", 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.293e-06 2024-09-15T00:09:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063121+0000", 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1068e-05 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.063127+0000", 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0989999999999998e-06 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068237+0000", 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051106720000000001 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068294+0000", 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6354000000000003e-05 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.068366+0000", 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2652999999999997e-05 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070616+0000", 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022500749999999998 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070971+0000", 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035480700000000001 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.070997+0000", 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5905999999999999e-05 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.071015+0000", 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8445e-05 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.373113+0000", 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.374874595000001, 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006661167, 2024-09-15T00:09:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 139 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373113+0000", 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373113+0000", 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373113+0000", 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6199999999999998e-07 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373120+0000", 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1099999999999997e-06 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373129+0000", 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0679999999999997e-06 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373137+0000", 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0790000000000001e-06 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.376789+0000", 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036521850000000001 2024-09-15T00:09:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.376837+0000", 2024-09-15T00:09:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8538000000000002e-05 2024-09-15T00:09:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.376901+0000", 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3958e-05 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.377773+0000", 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087204899999999996 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379724+0000", 2024-09-15T00:09:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001950481 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379755+0000", 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1618000000000001e-05 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379774+0000", 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8618999999999999e-05 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.373146+0000", 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.374841823000001, 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066998939999999996, 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 140 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373146+0000", 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373146+0000", 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373146+0000", 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000002e-07 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373216+0000", 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9774e-05 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373223+0000", 2024-09-15T00:09:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4939999999999997e-06 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373228+0000", 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9779999999999996e-06 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.377147+0000", 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003918464 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.377192+0000", 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5470999999999999e-05 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.377263+0000", 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1110000000000002e-05 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378345+0000", 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010819359999999999 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379804+0000", 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001458595 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379828+0000", 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4172000000000002e-05 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379846+0000", 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7626999999999999e-05 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.373237+0000", 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.374750683999999, 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066716279999999998, 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 141 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373237+0000", 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373237+0000", 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373237+0000", 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8099999999999998e-07 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373243+0000", 2024-09-15T00:09:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5060000000000003e-06 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373251+0000", 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.072e-06 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.373256+0000", 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0309999999999998e-06 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378036+0000", 2024-09-15T00:09:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047805390000000003 2024-09-15T00:09:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378078+0000", 2024-09-15T00:09:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1504000000000001e-05 2024-09-15T00:09:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378145+0000", 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6908000000000004e-05 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.378600+0000", 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00045539300000000001 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379866+0000", 2024-09-15T00:09:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012655539999999999 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379892+0000", 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6154999999999999e-05 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.379908+0000", 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6585000000000001e-05 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.797279+0000", 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.950708559999999, 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083034489999999992, 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 158 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797279+0000", 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797279+0000", 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797279+0000", 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2000000000000001e-07 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797311+0000", 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1927000000000001e-05 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797316+0000", 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4229999999999996e-06 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797323+0000", 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.438e-06 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.799837+0000", 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025140570000000001 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.799896+0000", 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8644999999999998e-05 2024-09-15T00:09:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.799982+0000", 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6878000000000004e-05 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.801206+0000", 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012232549999999999 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805493+0000", 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042870260000000002 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805543+0000", 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9728000000000002e-05 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805582+0000", 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9851999999999997e-05 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4174.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:45.797328+0000", 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.950659068, 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083428370000000005, 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4174", 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/3096760296", 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 159 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797328+0000", 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797328+0000", 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797329+0000", 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7399999999999999e-07 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797331+0000", 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.131e-06 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797334+0000", 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2540000000000001e-06 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.797337+0000", 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6299999999999998e-06 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.800437+0000", 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031001940000000001 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.800498+0000", 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0965999999999999e-05 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.800606+0000", 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107765 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.801437+0000", 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083154300000000002 2024-09-15T00:09:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805618+0000", 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041806769999999998 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805648+0000", 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9674000000000001e-05 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:45.805671+0000", 2024-09-15T00:09:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3728999999999999e-05 2024-09-15T00:09:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:05.876 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:05.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:05.878 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:05.876+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:05.963 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:09:05.966 INFO:teuthology.orchestra.run.smithi027.stderr:set 3.1e pg_upmap_items mapping to [3->4,0->1] 2024-09-15T00:09:05.978 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:05.976+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:06.031 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:845 3.1a 3.12cd873a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.030653+0000", 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3184999999999998e-05, 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8338000000000006e-05, 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 845 2024-09-15T00:09:06.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.030653+0000", 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.030653+0000", 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.030654+0000", 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7400000000000002e-07 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.030658+0000", 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.092e-06 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.030669+0000", 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0984e-05 2024-09-15T00:09:06.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.030682+0000", 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2387e-05 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.030752+0000", 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0627e-05 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:09:06.035 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-15T00:09:06.036 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:06.037 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone1 from 0xda29200 (overlap 0xb00a00) to 0x9e6e00 2024-09-15T00:09:06.040 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:06.082 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:06.080+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:06.180 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:06.178+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:06.186 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:06.186 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:06.186 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:06.186 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:06.186 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:06.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:06.280 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:06.279+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:06.301 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:01.958736+0000", 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.340172226, 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028349870000000002, 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.958736+0000", 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.958736+0000", 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.958737+0000", 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4100000000000001e-06 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.958745+0000", 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8259999999999993e-06 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.958757+0000", 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1554e-05 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.958768+0000", 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1126e-05 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.958811+0000", 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2939999999999999e-05 2024-09-15T00:09:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.958816+0000", 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.960396+0000", 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015807309999999999 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.960463+0000", 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999994e-05 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.960564+0000", 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101683 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.961468+0000", 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090390399999999997 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.961481+0000", 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2425999999999999e-05 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.961529+0000", 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8260000000000002e-05 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.961571+0000", 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2089000000000003e-05 2024-09-15T00:09:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4393.0:10 3.1c e23/20 3:3831950c:::rbd_directory:head v 23'1, mlcod=0'0)", 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:01.964400+0000", 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3345082499999998, 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002518209, 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964400+0000", 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964400+0000", 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964400+0000", 2024-09-15T00:09:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964406+0000", 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3680000000000001e-06 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964416+0000", 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0382e-05 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964422+0000", 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3509999999999998e-06 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964477+0000", 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4719999999999998e-05 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964502+0000", 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4765000000000001e-05 2024-09-15T00:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.966879+0000", 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023773309999999999 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.966918+0000", 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8952000000000002e-05 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1107 3.1a 3:5ce1b348:::rbd_header.1129feb48f4b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:05.966074+0000", 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.33283467, 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061400020000000003, 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1107 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.966074+0000", 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.966074+0000", 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.966075+0000", 2024-09-15T00:09:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9400000000000001e-06 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.966084+0000", 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6430000000000001e-06 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.966102+0000", 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8151999999999998e-05 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.966111+0000", 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6170000000000007e-06 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.966176+0000", 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4620000000000001e-05 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.966184+0000", 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1669999999999999e-06 2024-09-15T00:09:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.969962+0000", 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037781680000000002 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.970023+0000", 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1204000000000006e-05 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.970273+0000", 2024-09-15T00:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024973399999999999 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.971456+0000", 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001183258 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.972138+0000", 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068227999999999995 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.972184+0000", 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5630000000000002e-05 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:05.972214+0000", 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9589000000000001e-05 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:863 3.1a 3:5ce1b348:::rbd_header.1129feb48f4b:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.042324+0000", 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.25658436000000001, 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021017980000000002, 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 863 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042324+0000", 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042324+0000", 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042325+0000", 2024-09-15T00:09:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.043e-06 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042330+0000", 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1170000000000003e-06 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042339+0000", 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.918e-06 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042356+0000", 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6912e-05 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044293+0000", 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001937357 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044349+0000", 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5940000000000003e-05 2024-09-15T00:09:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044426+0000", 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6511e-05 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1158 3.1a 3:5ce1b348:::rbd_header.1129feb48f4b:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.042512+0000", 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.25639604999999999, 2024-09-15T00:09:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020716200000000001, 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042512+0000", 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042512+0000", 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042513+0000", 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1800000000000005e-07 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042517+0000", 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1200000000000004e-06 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042527+0000", 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3959999999999995e-06 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042533+0000", 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044471+0000", 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019381559999999999 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044525+0000", 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3415999999999998e-05 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044584+0000", 2024-09-15T00:09:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9020000000000001e-05 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:864 3.1a 3:5ce1b348:::rbd_header.1129feb48f4b:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.042542+0000", 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.25636579300000001, 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021767919999999999, 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 864 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042542+0000", 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042542+0000", 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042544+0000", 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3009999999999999e-06 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042549+0000", 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042560+0000", 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1447000000000001e-05 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042567+0000", 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1670000000000002e-06 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044618+0000", 2024-09-15T00:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002050685 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044657+0000", 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8435e-05 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044719+0000", 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2615999999999998e-05 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1159 3.1a 3:5ce1b348:::rbd_header.1129feb48f4b:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.042714+0000", 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.25619373899999998, 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021188119999999999, 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042714+0000", 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042714+0000", 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042715+0000", 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9800000000000003e-07 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042720+0000", 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5850000000000001e-06 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042726+0000", 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7640000000000002e-06 2024-09-15T00:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042730+0000", 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4100000000000001e-06 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044745+0000", 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020153969999999999 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044783+0000", 2024-09-15T00:09:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7481000000000002e-05 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044833+0000", 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0476999999999998e-05 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:865 3.1a 3:5ce1b348:::rbd_header.1129feb48f4b:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.042721+0000", 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.25618726600000002, 2024-09-15T00:09:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022434310000000002, 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 865 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042721+0000", 2024-09-15T00:09:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042721+0000", 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042722+0000", 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0890000000000001e-06 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042726+0000", 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0749999999999999e-06 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042735+0000", 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6659999999999997e-06 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.042742+0000", 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8279999999999997e-06 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044858+0000", 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002116522 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044896+0000", 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7858999999999998e-05 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.044964+0000", 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8392000000000005e-05 2024-09-15T00:09:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1162 3.1c e26/20 3:3c0bd203:::rbd_object_map.1129feb48f4b:head v 26'101, mlcod=26'98)", 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.046378+0000", 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.25253013699999999, 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020681760000000001, 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046378+0000", 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046378+0000", 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046379+0000", 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-07 2024-09-15T00:09:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046383+0000", 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999997e-06 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046391+0000", 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9840000000000001e-06 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046395+0000", 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1350000000000002e-06 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046433+0000", 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7929999999999998e-05 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046445+0000", 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1732999999999999e-05 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.048412+0000", 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001967749 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.048446+0000", 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3986999999999998e-05 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1164 3.1c e26/20 3:3c0bd203:::rbd_object_map.1129feb48f4b:head v 26'103, mlcod=26'98)", 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.047491+0000", 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.25141737800000002, 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020977180000000002, 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047491+0000", 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047491+0000", 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047492+0000", 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5799999999999999e-07 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047497+0000", 2024-09-15T00:09:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.789e-06 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047509+0000", 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1263e-05 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047514+0000", 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4070000000000001e-06 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047555+0000", 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1208999999999998e-05 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047571+0000", 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5736e-05 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049552+0000", 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019807990000000001 2024-09-15T00:09:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049589+0000", 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6857000000000003e-05 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1166 3.1c e26/20 3:3c0bd203:::rbd_object_map.1129feb48f4b:head v 26'105, mlcod=26'98)", 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.049161+0000", 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24974706899999999, 2024-09-15T00:09:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026065329999999999, 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049161+0000", 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049161+0000", 2024-09-15T00:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049162+0000", 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049166+0000", 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.685e-06 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049174+0000", 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8259999999999993e-06 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049181+0000", 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049224+0000", 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439000000000002e-05 2024-09-15T00:09:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049236+0000", 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2364999999999999e-05 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051725+0000", 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002488994 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051768+0000", 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2298000000000001e-05 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1174 3.1a 3:5d77dad2:::rbd_data.1129feb48f4b.0000000000000001:head [stat,truncate 917504] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.049215+0000", 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24969292600000001, 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028796130000000001, 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049215+0000", 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049215+0000", 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049216+0000", 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4900000000000005e-07 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049219+0000", 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9979999999999999e-06 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049228+0000", 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9830000000000002e-06 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049236+0000", 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7670000000000003e-06 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049311+0000", 2024-09-15T00:09:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4770000000000004e-05 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049395+0000", 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4534000000000001e-05 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049558+0000", 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016240699999999999 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050410+0000", 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085190099999999996 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052012+0000", 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001602509 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052056+0000", 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4217e-05 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052095+0000", 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8677999999999998e-05 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1173 3.1b e26/20 3:dc04e979:::rbd_data.1129feb48f4b.0000000000000000:head v 26'5, mlcod=24'2)", 2024-09-15T00:09:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.049478+0000", 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.249430281, 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00258355, 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049478+0000", 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049478+0000", 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049478+0000", 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4699999999999998e-07 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049483+0000", 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3619999999999999e-06 2024-09-15T00:09:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049497+0000", 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4399999999999999e-05 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049503+0000", 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0700000000000003e-06 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049556+0000", 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3455999999999998e-05 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049574+0000", 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7969999999999999e-05 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052019+0000", 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024448629999999998 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052061+0000", 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2181999999999997e-05 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1175 3.e e26/20 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:head v 26'7, mlcod=25'4)", 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.049770+0000", 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24913781300000001, 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023165540000000002, 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049770+0000", 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049770+0000", 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049771+0000", 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0399999999999996e-07 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049776+0000", 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8760000000000004e-06 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049784+0000", 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9279999999999993e-06 2024-09-15T00:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049789+0000", 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5450000000000003e-06 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049839+0000", 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9886999999999999e-05 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049859+0000", 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9749e-05 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052038+0000", 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021790920000000001 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052087+0000", 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8973e-05 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1167 3.1c e26/20 3:3c0bd203:::rbd_object_map.1129feb48f4b:head v 26'106, mlcod=26'101)", 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.049809+0000", 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24909894499999999, 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023258049999999998, 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049809+0000", 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049809+0000", 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049810+0000", 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:09:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049813+0000", 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2629999999999999e-06 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049819+0000", 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1800000000000001e-06 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049825+0000", 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4450000000000004e-06 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049863+0000", 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8881999999999998e-05 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049888+0000", 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4562e-05 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052098+0000", 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022104479999999998 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052135+0000", 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6683000000000001e-05 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1169 3.1c e26/20 3:3c0bd203:::rbd_object_map.1129feb48f4b:head v 26'108, mlcod=26'103)", 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.051298+0000", 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24761027299999999, 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040006529999999998, 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051298+0000", 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051298+0000", 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051299+0000", 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7100000000000001e-07 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051305+0000", 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4439999999999996e-06 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051317+0000", 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1984000000000001e-05 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051324+0000", 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.562e-06 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051370+0000", 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6495000000000001e-05 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051390+0000", 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0112999999999999e-05 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055257+0000", 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038665409999999998 2024-09-15T00:09:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055299+0000", 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1743000000000003e-05 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1170 3.1c e26/20 3:3c0bd203:::rbd_object_map.1129feb48f4b:head v 26'109, mlcod=26'103)", 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.052086+0000", 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.246822035, 2024-09-15T00:09:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032467519999999999, 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052086+0000", 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052086+0000", 2024-09-15T00:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052087+0000", 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.06e-07 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052092+0000", 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.22e-06 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052102+0000", 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6490000000000003e-06 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052118+0000", 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6370000000000001e-05 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052191+0000", 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2663000000000005e-05 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052208+0000", 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6821e-05 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055304+0000", 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030963919999999999 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055333+0000", 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9031e-05 2024-09-15T00:09:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1171 3.1c e26/20 3:3c0bd203:::rbd_object_map.1129feb48f4b:head v 26'110, mlcod=26'103)", 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.052762+0000", 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24614619200000001, 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026004930000000002, 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052762+0000", 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052762+0000", 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052763+0000", 2024-09-15T00:09:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6000000000000004e-07 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052766+0000", 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8589999999999998e-06 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052775+0000", 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5779999999999999e-06 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052779+0000", 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2969999999999997e-06 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052813+0000", 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3633000000000001e-05 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052824+0000", 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1537e-05 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055334+0000", 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025097330000000001 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055363+0000", 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8296000000000001e-05 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1184 3.3 3:c5ea53fe:::rbd_data.1129feb48f4b.000000000000000b:head [stat,truncate 786432] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.055996+0000", 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.242912194, 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035901700000000002, 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1184 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055996+0000", 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055996+0000", 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055997+0000", 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5899999999999995e-07 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056002+0000", 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9309999999999999e-06 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056013+0000", 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1026000000000001e-05 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056020+0000", 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8519999999999998e-06 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056059+0000", 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9066000000000002e-05 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056107+0000", 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7902999999999999e-05 2024-09-15T00:09:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056241+0000", 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013414199999999999 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.057389+0000", 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011481099999999999 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.059526+0000", 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002137089 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.059560+0000", 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4020000000000003e-05 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.059586+0000", 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6271999999999999e-05 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4468.0:1182 3.e e26/20 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:head v 26'9, mlcod=25'5)", 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.056125+0000", 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24278344600000001, 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024463319999999998, 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056125+0000", 2024-09-15T00:09:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056125+0000", 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056126+0000", 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3600000000000003e-07 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056131+0000", 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7910000000000001e-06 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056141+0000", 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7270000000000002e-06 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056151+0000", 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9229999999999997e-06 2024-09-15T00:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056201+0000", 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9999e-05 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056222+0000", 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0557999999999999e-05 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.058538+0000", 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002315994 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.058571+0000", 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3603999999999999e-05 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.403 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:06.404 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-15T00:09:06.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:06.406 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:06.403+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:06.433 INFO:teuthology.orchestra.run.smithi092.stdout:335 write 0x145ec00 thru 0x146844d (0x984e bytes) 2024-09-15T00:09:06.434 INFO:teuthology.orchestra.run.smithi092.stdout:336 read 0x778f600 thru 0x7791a63 (0x2464 bytes) 2024-09-15T00:09:06.434 INFO:teuthology.orchestra.run.smithi092.stdout:340 punch from 0x5307200 to 0x5309e00, (0x2c00 bytes) 2024-09-15T00:09:06.456 INFO:teuthology.orchestra.run.smithi092.stdout:341 read 0x39f0600 thru 0x39f8b1d (0x851e bytes) 2024-09-15T00:09:06.456 INFO:teuthology.orchestra.run.smithi092.stdout:344 write 0x87abe00 thru 0x87adf63 (0x2164 bytes) 2024-09-15T00:09:06.456 INFO:teuthology.orchestra.run.smithi092.stdout:349 read 0x90fc00 thru 0x91a7b5 (0xabb6 bytes) 2024-09-15T00:09:06.456 INFO:teuthology.orchestra.run.smithi092.stdout:350 write 0xd9c9c00 thru 0xd9d2188 (0x8589 bytes) 2024-09-15T00:09:06.457 INFO:teuthology.orchestra.run.smithi092.stdout:352 read 0xabd5600 thru 0xabdbfaa (0x69ab bytes) 2024-09-15T00:09:06.457 INFO:teuthology.orchestra.run.smithi092.stdout:355 write 0x2911000 thru 0x291ecab (0xdcac bytes) 2024-09-15T00:09:06.457 INFO:teuthology.orchestra.run.smithi092.stdout:356 write 0x1b56000 thru 0x1b610ff (0xb100 bytes) 2024-09-15T00:09:06.470 INFO:teuthology.orchestra.run.smithi092.stdout:358 trunc from 0xda29200 to 0x358a600 2024-09-15T00:09:06.505 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:06.503+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:06.511 INFO:teuthology.orchestra.run.smithi092.stdout:360 write 0x86ec00 thru 0x87a5f8 (0xb9f9 bytes) 2024-09-15T00:09:06.513 INFO:teuthology.orchestra.run.smithi092.stdout:362 punch from 0x793000 to 0x79d200, (0xa200 bytes) 2024-09-15T00:09:06.514 INFO:teuthology.orchestra.run.smithi092.stdout:363 punch from 0x2237e00 to 0x223a600, (0x2800 bytes) 2024-09-15T00:09:06.520 INFO:teuthology.orchestra.run.smithi092.stdout:365 trunc from 0x358a600 to 0x8073000 2024-09-15T00:09:06.546 INFO:teuthology.orchestra.run.smithi092.stdout:367 trunc from 0x8073000 to 0x93c3c00 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:831 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 65536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.559266+0000", 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00036453800000000003, 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038700900000000001, 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:06.561 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 831 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559266+0000", 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559266+0000", 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559266+0000", 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5400000000000002e-07 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559269+0000", 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4269999999999998e-06 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559276+0000", 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0759999999999997e-06 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559281+0000", 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2859999999999999e-06 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559523+0000", 2024-09-15T00:09:06.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024158299999999999 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559614+0000", 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1620999999999996e-05 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:832 3.e 3.5673a2e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.559289+0000", 2024-09-15T00:09:06.565 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.000341393, 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040315000000000001, 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 832 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559289+0000", 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559289+0000", 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559290+0000", 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0200000000000003e-07 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559292+0000", 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.198e-06 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559298+0000", 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2759999999999997e-06 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559309+0000", 2024-09-15T00:09:06.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.083e-05 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:833 3.e 3.5673a2e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.559317+0000", 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00031388800000000001, 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040323399999999998, 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:06.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 833 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559317+0000", 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559317+0000", 2024-09-15T00:09:06.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559317+0000", 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559319+0000", 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559326+0000", 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1659999999999997e-06 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559332+0000", 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.242e-06 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:834 3.e 3.5673a2e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.559338+0000", 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00029251400000000001, 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00042769800000000001, 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 834 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559338+0000", 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559338+0000", 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559338+0000", 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.39e-07 2024-09-15T00:09:06.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559348+0000", 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4860000000000003e-06 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559354+0000", 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4559999999999997e-06 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559361+0000", 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3249999999999996e-06 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:835 3.e 3.5673a2e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.559368+0000", 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00026246599999999999, 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00045264200000000001, 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.576 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 835 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559368+0000", 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559368+0000", 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559369+0000", 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559370+0000", 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.95e-06 2024-09-15T00:09:06.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559377+0000", 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2929999999999997e-06 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559381+0000", 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:836 3.e 3.5673a2e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.559389+0000", 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00024182899999999999, 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00048525, 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 836 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559389+0000", 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559389+0000", 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559389+0000", 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4099999999999999e-07 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559391+0000", 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8339999999999999e-06 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559398+0000", 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9489999999999999e-06 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559403+0000", 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0679999999999996e-06 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:837 3.e 3.5673a2e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.559409+0000", 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00022152700000000001, 2024-09-15T00:09:06.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052238600000000001, 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 837 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559409+0000", 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559409+0000", 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559409+0000", 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559411+0000", 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8390000000000001e-06 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559420+0000", 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6410000000000008e-06 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.559425+0000", 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5099999999999998e-06 2024-09-15T00:09:06.586 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:09:06.587 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 7 2024-09-15T00:09:06.587 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:368 write 0xbed2600 thru 0xbed6a87 (0x4488 bytes) 2024-09-15T00:09:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:369 read 0x5bbba00 thru 0x5bc8c07 (0xd208 bytes) 2024-09-15T00:09:06.588 INFO:teuthology.orchestra.run.smithi092.stdout:370 punch from 0x2d12800 to 0x2d20200, (0xda00 bytes) 2024-09-15T00:09:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:371 write 0x647aa00 thru 0x648649f (0xbaa0 bytes) 2024-09-15T00:09:06.589 INFO:teuthology.orchestra.run.smithi092.stdout:372 write 0x7192600 thru 0x7192cbe (0x6bf bytes) 2024-09-15T00:09:06.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:06.595 INFO:teuthology.orchestra.run.smithi092.stdout:377 write 0xe00bc00 thru 0xe01581a (0x9c1b bytes) 2024-09-15T00:09:06.596 INFO:teuthology.orchestra.run.smithi092.stdout:378 read 0x1fcb200 thru 0x1fd21ea (0x6feb bytes) 2024-09-15T00:09:06.597 INFO:teuthology.orchestra.run.smithi092.stdout:380 clone 3 order 20 su 8192 sc 10 2024-09-15T00:09:06.605 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:06.603+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:06.707 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:06.705+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:06.730 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:06.731 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:06.731 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:06.731 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:06.731 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:06.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:06.807 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:06.805+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:06.849 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:51.835134+0000", 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.010715428999999, 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0071710940000000003, 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4208", 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.27:0/140383429", 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.835134+0000", 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.835134+0000", 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.835136+0000", 2024-09-15T00:09:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9429999999999999e-06 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.835145+0000", 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.217e-06 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.835161+0000", 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5787999999999999e-05 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.835171+0000", 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7380000000000006e-06 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.835243+0000", 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2099000000000002e-05 2024-09-15T00:09:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.835250+0000", 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6830000000000002e-06 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.840347+0000", 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050970809999999998 2024-09-15T00:09:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.840518+0000", 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017138200000000001 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.840676+0000", 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015745599999999999 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.841512+0000", 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083590300000000004 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.842212+0000", 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070053599999999998 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.842262+0000", 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9928e-05 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:51.842305+0000", 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3340000000000002e-05 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:08:52.835524+0000", 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.010326130999999, 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056066359999999999, 2024-09-15T00:09:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4208", 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.27:0/140383429", 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.835524+0000", 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.835524+0000", 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.835526+0000", 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0169999999999999e-06 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.835537+0000", 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1053000000000001e-05 2024-09-15T00:09:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.835555+0000", 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8312e-05 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.835567+0000", 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2148e-05 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.835645+0000", 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7574e-05 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.835654+0000", 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9849999999999995e-06 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.839247+0000", 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035935479999999998 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.839346+0000", 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8981000000000007e-05 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.839445+0000", 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8783000000000006e-05 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.840270+0000", 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00082484699999999997 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.841030+0000", 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076053899999999996 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.841089+0000", 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8873999999999999e-05 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:08:52.841130+0000", 2024-09-15T00:09:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0975e-05 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:01.962495+0000", 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8833548740000001, 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046410150000000001, 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 10 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.962495+0000", 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.962495+0000", 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.962496+0000", 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.285e-06 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.962500+0000", 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2379999999999999e-06 2024-09-15T00:09:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.962508+0000", 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8189999999999992e-06 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.962517+0000", 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4819999999999994e-06 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.962564+0000", 2024-09-15T00:09:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7445999999999999e-05 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.962568+0000", 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964023+0000", 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014549770000000001 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964156+0000", 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013343099999999999 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964305+0000", 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014818899999999999 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.964887+0000", 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00058267599999999996 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.967039+0000", 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00215168 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.967095+0000", 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6038000000000001e-05 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:01.967136+0000", 2024-09-15T00:09:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0924000000000003e-05 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1163 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045469+0000", 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80038073600000004, 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037602479999999999, 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1163 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045469+0000", 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045469+0000", 2024-09-15T00:09:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045469+0000", 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.28e-07 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045471+0000", 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045476+0000", 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9609999999999996e-06 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045479+0000", 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9100000000000001e-06 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046588+0000", 2024-09-15T00:09:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011096750000000001 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046644+0000", 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5819000000000002e-05 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.046841+0000", 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000197152 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.048999+0000", 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021577079999999999 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049155+0000", 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015562700000000001 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049192+0000", 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7098999999999998e-05 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049229+0000", 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7450000000000002e-05 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1164 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045482+0000", 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80036738799999996, 2024-09-15T00:09:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053711200000000001, 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1164 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045482+0000", 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045482+0000", 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045483+0000", 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2499999999999999e-07 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045485+0000", 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5859999999999999e-06 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045488+0000", 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.219e-06 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045491+0000", 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6800000000000002e-06 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047144+0000", 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016532230000000001 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047195+0000", 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0334999999999998e-05 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047432+0000", 2024-09-15T00:09:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023765099999999999 2024-09-15T00:09:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050047+0000", 2024-09-15T00:09:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026147700000000002 2024-09-15T00:09:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050795+0000", 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074771199999999999 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050827+0000", 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2499999999999997e-05 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050853+0000", 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6219000000000001e-05 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1165 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045495+0000", 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80035469699999995, 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054651910000000003, 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1165 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045495+0000", 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045495+0000", 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045495+0000", 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045497+0000", 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4300000000000001e-06 2024-09-15T00:09:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045504+0000", 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2749999999999998e-06 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045507+0000", 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0680000000000001e-06 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047714+0000", 2024-09-15T00:09:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022068729999999998 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047782+0000", 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7609999999999998e-05 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.047977+0000", 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195742 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050863+0000", 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028858410000000001 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050897+0000", 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3488999999999997e-05 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050934+0000", 2024-09-15T00:09:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7258999999999997e-05 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050960+0000", 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6290000000000001e-05 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1166 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045513+0000", 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80033694099999997, 2024-09-15T00:09:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076511050000000001, 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1166 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045513+0000", 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045513+0000", 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045513+0000", 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6e-07 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045515+0000", 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7430000000000001e-06 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045519+0000", 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3470000000000001e-06 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045522+0000", 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6819999999999999e-06 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.048245+0000", 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027235689999999999 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.048328+0000", 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2531e-05 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.048620+0000", 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029238000000000002 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053070+0000", 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044497809999999999 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053109+0000", 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8994e-05 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053140+0000", 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1260999999999998e-05 2024-09-15T00:09:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053164+0000", 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3557000000000002e-05 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1167 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045525+0000", 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80032458299999998, 2024-09-15T00:09:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077337370000000001, 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1167 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045525+0000", 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045525+0000", 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045525+0000", 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2399999999999999e-07 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045527+0000", 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.359e-06 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045533+0000", 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1659999999999999e-06 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045536+0000", 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999999e-06 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049297+0000", 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037615420000000001 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049354+0000", 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7200000000000001e-05 2024-09-15T00:09:06.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.049742+0000", 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038754600000000001 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053197+0000", 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034550010000000001 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053205+0000", 2024-09-15T00:09:06.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0749999999999998e-06 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053233+0000", 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7735000000000001e-05 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053259+0000", 2024-09-15T00:09:06.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6179000000000001e-05 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1168 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045539+0000", 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80031059400000004, 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078158429999999994, 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1168 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045539+0000", 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045539+0000", 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045539+0000", 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1500000000000001e-07 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045541+0000", 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.708e-06 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045545+0000", 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.63e-06 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045547+0000", 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.469e-06 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050094+0000", 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045463700000000001 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050177+0000", 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3158999999999994e-05 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050434+0000", 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025709799999999998 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053265+0000", 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002831376 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053297+0000", 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1467000000000002e-05 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053334+0000", 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7174e-05 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053355+0000", 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1177000000000001e-05 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1169 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045550+0000", 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80029941999999998, 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0099760439999999999, 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1169 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045550+0000", 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045550+0000", 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045551+0000", 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.11e-07 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045557+0000", 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045561+0000", 2024-09-15T00:09:06.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3669999999999999e-06 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045565+0000", 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2669999999999998e-06 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.050990+0000", 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054249490000000001 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051039+0000", 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8949999999999997e-05 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051238+0000", 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019956800000000001 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055212+0000", 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039742019999999996 2024-09-15T00:09:06.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055462+0000", 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024914000000000002 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055491+0000", 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9626000000000001e-05 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055526+0000", 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5187e-05 2024-09-15T00:09:06.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1170 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045568+0000", 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80028145399999995, 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010108783, 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1170 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045568+0000", 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045568+0000", 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045568+0000", 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.29e-07 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045570+0000", 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-15T00:09:06.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045573+0000", 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999999e-06 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045576+0000", 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4700000000000001e-06 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051617+0000", 2024-09-15T00:09:06.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006041383 2024-09-15T00:09:06.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051705+0000", 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7348000000000005e-05 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052017+0000", 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000312789 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055223+0000", 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003205374 2024-09-15T00:09:06.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055580+0000", 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000356862 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055639+0000", 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9172000000000002e-05 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055677+0000", 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8235999999999998e-05 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1171 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045581+0000", 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80026873200000004, 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010234055000000001, 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.923 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1171 2024-09-15T00:09:06.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.923 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045581+0000", 2024-09-15T00:09:06.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045581+0000", 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045581+0000", 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2999999999999999e-07 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045583+0000", 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.517e-06 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045586+0000", 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0989999999999999e-06 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045588+0000", 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5859999999999999e-06 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052357+0000", 2024-09-15T00:09:06.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067687939999999999 2024-09-15T00:09:06.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052438+0000", 2024-09-15T00:09:06.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0728000000000004e-05 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052711+0000", 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027272100000000002 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055228+0000", 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025168970000000001 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055742+0000", 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051467500000000003 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055791+0000", 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8950999999999999e-05 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055815+0000", 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3856999999999999e-05 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.928 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1172 3.1c 3:3c0bd203:::rbd_object_map.1129feb48f4b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.045592+0000", 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.80025817899999996, 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01034383, 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.929 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1172 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045592+0000", 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045592+0000", 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045592+0000", 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1e-07 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045596+0000", 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5839999999999996e-06 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045599+0000", 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1590000000000002e-06 2024-09-15T00:09:06.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.045603+0000", 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.18e-06 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053382+0000", 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077797329999999996 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053430+0000", 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7333000000000002e-05 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.053713+0000", 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028311800000000001 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055232+0000", 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015190119999999999 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055862+0000", 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00062978600000000002 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055894+0000", 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2397000000000002e-05 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055935+0000", 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1318e-05 2024-09-15T00:09:06.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1176 3.1c 3:3bc4bd9e:::rbd_data.1129feb48f4b.0000000000000003:head [stat,truncate 917504] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.051024+0000", 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.79482573099999998, 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049983240000000002, 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1176 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051024+0000", 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051024+0000", 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051025+0000", 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0100000000000004e-07 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051029+0000", 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051040+0000", 2024-09-15T00:09:06.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0352e-05 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051045+0000", 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2070000000000003e-06 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.054077+0000", 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030318459999999999 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.054149+0000", 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2835000000000003e-05 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.054303+0000", 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000153726 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055970+0000", 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016664900000000001 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055976+0000", 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.759e-06 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056002+0000", 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5347e-05 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.056022+0000", 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0531e-05 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4468.0:1177 3.5 3:a747d467:::rbd_data.1129feb48f4b.0000000000000004:head [stat,truncate 917504] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.051059+0000", 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.79479101299999999, 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041595039999999996, 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.941 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4468", 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1552213568", 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1177 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051059+0000", 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051059+0000", 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051059+0000", 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3599999999999999e-07 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051061+0000", 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9530000000000002e-06 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051066+0000", 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2739999999999998e-06 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051072+0000", 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.502e-06 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051110+0000", 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8297999999999998e-05 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051161+0000", 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0915999999999998e-05 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.051270+0000", 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010945700000000001 2024-09-15T00:09:06.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.052236+0000", 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00096517200000000004 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055121+0000", 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028856770000000001 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055177+0000", 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6137000000000001e-05 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.055218+0000", 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0781999999999997e-05 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:333 3.1d 3:bf1a1b4e:::rbd_data.1129aa143e51.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 65536~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.444997+0000", 2024-09-15T00:09:06.947 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.4008525, 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049794549999999998, 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 333 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.444997+0000", 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.444997+0000", 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.444998+0000", 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0610000000000001e-06 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.445027+0000", 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9122e-05 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.445038+0000", 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0525e-05 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.445043+0000", 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3569999999999997e-06 2024-09-15T00:09:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.445084+0000", 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0284999999999998e-05 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.445161+0000", 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6970000000000003e-05 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.445289+0000", 2024-09-15T00:09:06.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012891500000000001 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.447038+0000", 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748915 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.449896+0000", 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028576180000000001 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.449939+0000", 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2885e-05 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.449977+0000", 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7802000000000002e-05 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.953 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:439 3.1c 3:3ee4b00e:::rbd_data.1129feb48f4b.000000000000000f:4 [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.470433+0000", 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.37541703599999998, 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037441229999999998, 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 439 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470433+0000", 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470433+0000", 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470433+0000", 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3699999999999999e-07 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470438+0000", 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3770000000000004e-06 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470442+0000", 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7960000000000002e-06 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470445+0000", 2024-09-15T00:09:06.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9050000000000001e-06 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.474049+0000", 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036035339999999998 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.474115+0000", 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6827000000000002e-05 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.474177+0000", 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1446999999999996e-05 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:440 3.1c 3:3ee4b00e:::rbd_data.1129feb48f4b.000000000000000f:4 [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.470449+0000", 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.37540084299999998, 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003856351, 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:06.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 440 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470449+0000", 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470449+0000", 2024-09-15T00:09:06.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470449+0000", 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1e-07 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470450+0000", 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0240000000000001e-06 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470453+0000", 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9059999999999998e-06 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.470455+0000", 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4830000000000002e-06 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.474201+0000", 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00374559 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.474247+0000", 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5921999999999998e-05 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.474305+0000", 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8216e-05 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.962 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:989 3.1c 3:3c7c539d:::rbd_data.1129aa143e51.000000000000005b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 16384~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.579577+0000", 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.26627244799999999, 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038434179999999999, 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 989 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579577+0000", 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579577+0000", 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579578+0000", 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7999999999999993e-07 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579592+0000", 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3740000000000001e-05 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579603+0000", 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.059e-05 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579608+0000", 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4410000000000001e-06 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579658+0000", 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9549000000000003e-05 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579783+0000", 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012546700000000001 2024-09-15T00:09:06.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579971+0000", 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000187935 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.580588+0000", 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00061671000000000002 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.583319+0000", 2024-09-15T00:09:06.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002731542 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.583378+0000", 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9038e-05 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.583421+0000", 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2425999999999997e-05 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4486.0:998 3.e 3:75def05e:::rbd_data.1129aa143e51.00000000000000c8:head [set-alloc-hint object_size 524288 write_size 524288,write 109056~5632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:06.579748+0000", 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.266101278, 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068320330000000004, 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4486", 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 998 2024-09-15T00:09:06.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579748+0000", 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579748+0000", 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:06.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579749+0000", 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.96e-07 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579756+0000", 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8349999999999998e-06 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579761+0000", 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1019999999999996e-06 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.579764+0000", 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7179999999999998e-06 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.581008+0000", 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012435479999999999 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.581071+0000", 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3230000000000003e-05 2024-09-15T00:09:06.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.581154+0000", 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2713000000000006e-05 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.582945+0000", 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001790699 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.586514+0000", 2024-09-15T00:09:06.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035689559999999999 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.586555+0000", 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1301999999999999e-05 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:06.586580+0000", 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5633999999999999e-05 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.974 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:06.975 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:06.975 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:06.976 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone2 from 0xe015a00 (overlap 0x358a600) to 0x2ad5000 2024-09-15T00:09:06.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:06.977 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:06.974+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:07.077 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:07.076+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:07.086 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:07.087 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:07.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:07.179 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:07.177+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:07.209 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:07.209 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:07.214 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-15T00:09:07.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:07.302 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:07.300+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:07.304 INFO:teuthology.orchestra.run.smithi092.stdout:383 write 0x48f2400 thru 0x48f5274 (0x2e75 bytes) 2024-09-15T00:09:07.304 INFO:teuthology.orchestra.run.smithi092.stdout:385 punch from 0x82c2400 to 0x82cc000, (0x9c00 bytes) 2024-09-15T00:09:07.335 INFO:teuthology.orchestra.run.smithi092.stdout:389 trunc from 0xe015a00 to 0x1543000 2024-09-15T00:09:07.380 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:07.379+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:07.385 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:01.967485+0000", 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.4145025550000003, 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0081594440000000001, 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 11 2024-09-15T00:09:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.967485+0000", 2024-09-15T00:09:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.967485+0000", 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.967487+0000", 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4159999999999998e-06 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.967503+0000", 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5627000000000001e-05 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.967533+0000", 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9906000000000001e-05 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.967550+0000", 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7102e-05 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.967716+0000", 2024-09-15T00:09:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016598900000000001 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.967732+0000", 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5987000000000001e-05 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.969665+0000", 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019331629999999999 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.969749+0000", 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3666999999999997e-05 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.969914+0000", 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016483700000000001 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.971438+0000", 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015243889999999999 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.975580+0000", 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004142326 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.975618+0000", 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:01.975645+0000", 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6658000000000001e-05 2024-09-15T00:09:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:356 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.470463+0000", 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.91152482400000001, 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011867077, 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 356 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.470463+0000", 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.470463+0000", 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.470463+0000", 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8700000000000005e-07 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.470468+0000", 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3529999999999997e-06 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.470478+0000", 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9450000000000005e-06 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.470487+0000", 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7679999999999997e-06 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.473791+0000", 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003304366 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.473838+0000", 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7515000000000002e-05 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474185+0000", 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00034652000000000001 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.479665+0000", 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054804370000000003 2024-09-15T00:09:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482229+0000", 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0025632989999999998 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482294+0000", 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5753000000000006e-05 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482330+0000", 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5434e-05 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:451 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.474064+0000", 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.90792330499999996, 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0083601920000000007, 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 451 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474064+0000", 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474064+0000", 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474065+0000", 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.16e-07 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474070+0000", 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8629999999999999e-06 2024-09-15T00:09:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474082+0000", 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2431e-05 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474099+0000", 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6577000000000001e-05 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474509+0000", 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040980500000000001 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474573+0000", 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4670000000000003e-05 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474855+0000", 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028153800000000001 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.479697+0000", 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004842201 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482336+0000", 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026388610000000002 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482376+0000", 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9870000000000003e-05 2024-09-15T00:09:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482425+0000", 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8560000000000003e-05 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:452 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.474129+0000", 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.90785891200000002, 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0083492299999999992, 2024-09-15T00:09:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 452 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474129+0000", 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474129+0000", 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474129+0000", 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6499999999999999e-07 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474133+0000", 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7290000000000002e-06 2024-09-15T00:09:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474142+0000", 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4849999999999992e-06 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474147+0000", 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7729999999999996e-06 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475168+0000", 2024-09-15T00:09:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001020998 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475215+0000", 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6195000000000001e-05 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475525+0000", 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00031010599999999999 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482157+0000", 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0066319430000000004 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482431+0000", 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027470700000000001 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482457+0000", 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5664999999999999e-05 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482478+0000", 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0964000000000001e-05 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:453 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.474679+0000", 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.90730908300000002, 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078457720000000009, 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 453 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474679+0000", 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474679+0000", 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474679+0000", 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3900000000000004e-07 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474683+0000", 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7299999999999999e-06 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474690+0000", 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0709999999999997e-06 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.474694+0000", 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4309999999999996e-06 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475804+0000", 2024-09-15T00:09:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011093629999999999 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475885+0000", 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0834999999999994e-05 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476234+0000", 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000349814 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482212+0000", 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0059776830000000001 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482485+0000", 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027266399999999999 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482507+0000", 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2065999999999998e-05 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482524+0000", 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7476e-05 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:454 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.475314+0000", 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.90667337699999995, 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0095747620000000006, 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 454 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475314+0000", 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475314+0000", 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475315+0000", 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3500000000000005e-07 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475320+0000", 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:09:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475334+0000", 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4253999999999999e-05 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.475341+0000", 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9029999999999998e-06 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476567+0000", 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001226188 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476627+0000", 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0154e-05 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476911+0000", 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028386999999999998 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.482555+0000", 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.005643688 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484796+0000", 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022412500000000002 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484847+0000", 2024-09-15T00:09:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0498999999999999e-05 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484889+0000", 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2444999999999998e-05 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:455 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.476556+0000", 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.90543132800000004, 2024-09-15T00:09:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0084214330000000007, 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 455 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476556+0000", 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476556+0000", 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476557+0000", 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476561+0000", 2024-09-15T00:09:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7629999999999998e-06 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476568+0000", 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1049999999999997e-06 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476572+0000", 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3059999999999999e-06 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477761+0000", 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011890519999999999 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477814+0000", 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2882e-05 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.478036+0000", 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022212800000000001 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484174+0000", 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0061372589999999999 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484898+0000", 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00072477200000000005 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484945+0000", 2024-09-15T00:09:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6718000000000002e-05 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484978+0000", 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2731000000000002e-05 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:456 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.476640+0000", 2024-09-15T00:09:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.90534722499999998, 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0083926869999999994, 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 456 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476640+0000", 2024-09-15T00:09:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476640+0000", 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476641+0000", 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6400000000000001e-07 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476654+0000", 2024-09-15T00:09:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2969999999999999e-05 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476665+0000", 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1090999999999999e-05 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476678+0000", 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.307e-05 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.478405+0000", 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0017270689999999999 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.478453+0000", 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7153000000000001e-05 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.478694+0000", 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024147599999999999 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484210+0000", 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0055155029999999997 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484982+0000", 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00077245099999999995 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485008+0000", 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5726000000000001e-05 2024-09-15T00:09:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485033+0000", 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5313999999999999e-05 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:457 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.476780+0000", 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.90520773300000001, 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0083430589999999999, 2024-09-15T00:09:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 457 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476780+0000", 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476780+0000", 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476781+0000", 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4700000000000001e-07 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476785+0000", 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8750000000000002e-06 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476794+0000", 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6530000000000006e-06 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.476801+0000", 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1659999999999997e-06 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.478983+0000", 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021812839999999999 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.479035+0000", 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2256999999999998e-05 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.479315+0000", 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028039500000000001 2024-09-15T00:09:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484233+0000", 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004917467 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485040+0000", 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00080717199999999999 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485082+0000", 2024-09-15T00:09:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2131e-05 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485123+0000", 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0911999999999997e-05 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:458 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.477036+0000", 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.90495180799999997, 2024-09-15T00:09:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0081679950000000008, 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 458 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477036+0000", 2024-09-15T00:09:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477036+0000", 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477037+0000", 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1379999999999999e-06 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477041+0000", 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6880000000000001e-06 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477054+0000", 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3499000000000001e-05 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477060+0000", 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.229e-06 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.479725+0000", 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026649849999999999 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.479775+0000", 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9370000000000003e-05 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.479983+0000", 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020864500000000001 2024-09-15T00:09:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484262+0000", 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042784479999999998 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485131+0000", 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086868100000000005 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485170+0000", 2024-09-15T00:09:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9169999999999999e-05 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485204+0000", 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4141999999999999e-05 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:462 3.12 3:48352226:::rbd_data.1129aa143e51.00000000000001b6:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 56320~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.477916+0000", 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.904071611, 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0074113909999999998, 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 462 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477916+0000", 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477916+0000", 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477917+0000", 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.16e-07 2024-09-15T00:09:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477933+0000", 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5999999999999999e-05 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477943+0000", 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0409e-05 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.477950+0000", 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7039999999999997e-06 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.480228+0000", 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022782850000000001 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.480314+0000", 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5883000000000004e-05 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.480435+0000", 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000120314 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.484283+0000", 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038484359999999998 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485211+0000", 2024-09-15T00:09:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092802399999999995 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485266+0000", 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5306000000000002e-05 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.485327+0000", 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1113999999999998e-05 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:954 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.572991+0000", 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.808996928, 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076089139999999996, 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 954 2024-09-15T00:09:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.572991+0000", 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.572991+0000", 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.572991+0000", 2024-09-15T00:09:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3300000000000002e-07 2024-09-15T00:09:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.572995+0000", 2024-09-15T00:09:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6069999999999999e-06 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573004+0000", 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6419999999999996e-06 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573013+0000", 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6080000000000002e-06 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.574267+0000", 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012540520000000001 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.574356+0000", 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8998000000000004e-05 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.574760+0000", 2024-09-15T00:09:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040397399999999998 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.576802+0000", 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020420730000000002 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580536+0000", 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0037340659999999999 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580564+0000", 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7634999999999999e-05 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580600+0000", 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5626000000000001e-05 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:955 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.573023+0000", 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.808964397, 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0076689519999999997, 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 955 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573023+0000", 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573023+0000", 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573024+0000", 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4199999999999996e-07 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573027+0000", 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.004e-06 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573034+0000", 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1339999999999998e-06 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573039+0000", 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.215e-06 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.575270+0000", 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022311800000000001 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.575318+0000", 2024-09-15T00:09:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7494000000000003e-05 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.575532+0000", 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00021374100000000001 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.576863+0000", 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013314869999999999 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580606+0000", 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0037429270000000001 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580653+0000", 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7125e-05 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580692+0000", 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9103000000000002e-05 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:956 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.573051+0000", 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.80893695799999998, 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0077144739999999998, 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 956 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573051+0000", 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573051+0000", 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573051+0000", 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4199999999999996e-07 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573054+0000", 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9869999999999999e-06 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573061+0000", 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5030000000000002e-06 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573066+0000", 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.434e-06 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.575940+0000", 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002873422 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.576025+0000", 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5028e-05 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.576415+0000", 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039088799999999999 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.578907+0000", 2024-09-15T00:09:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024918399999999999 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580699+0000", 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0017915590000000001 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580731+0000", 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1655000000000001e-05 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580765+0000", 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4616000000000002e-05 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:957 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.573074+0000", 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.80891326100000005, 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.007840138, 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 957 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573074+0000", 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573074+0000", 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573075+0000", 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573077+0000", 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5019999999999999e-06 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573083+0000", 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1369999999999999e-06 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573088+0000", 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8339999999999998e-06 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.576906+0000", 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038173270000000001 2024-09-15T00:09:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.576962+0000", 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6477000000000002e-05 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.577202+0000", 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023955600000000001 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580771+0000", 2024-09-15T00:09:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00356954 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580816+0000", 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5311e-05 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580874+0000", 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7030000000000001e-05 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.580915+0000", 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1004999999999998e-05 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:958 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.573193+0000", 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.80879435499999996, 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010267691000000001, 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 958 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573193+0000", 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573193+0000", 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573194+0000", 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7199999999999998e-07 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573198+0000", 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0670000000000002e-06 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573207+0000", 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3349999999999996e-06 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573215+0000", 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.571e-06 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.577508+0000", 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042932589999999998 2024-09-15T00:09:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.577593+0000", 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4764000000000004e-05 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.577967+0000", 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037357500000000001 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.581458+0000", 2024-09-15T00:09:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003491311 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.583401+0000", 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001943146 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.583433+0000", 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2156000000000002e-05 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.583461+0000", 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7835e-05 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:961 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.573779+0000", 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.80820900600000001, 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0098423980000000005, 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 961 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573779+0000", 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573779+0000", 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573779+0000", 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2799999999999995e-07 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573785+0000", 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5729999999999998e-06 2024-09-15T00:09:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573795+0000", 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8069999999999994e-06 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.573802+0000", 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0219999999999999e-06 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.578345+0000", 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.004543559 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.578406+0000", 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0238000000000002e-05 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.578608+0000", 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020266199999999999 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.581461+0000", 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028528 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.583517+0000", 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020555310000000002 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.583576+0000", 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8987999999999998e-05 2024-09-15T00:09:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.583621+0000", 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5389999999999997e-05 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:963 3.12 3:4832bfd6:::rbd_object_map.1129aa143e51:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.574124+0000", 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.80786320199999995, 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0095848760000000009, 2024-09-15T00:09:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 963 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.574124+0000", 2024-09-15T00:09:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.574124+0000", 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.574125+0000", 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9899999999999999e-07 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.574129+0000", 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1439999999999996e-06 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.574140+0000", 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0216e-05 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.574147+0000", 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5739999999999998e-06 2024-09-15T00:09:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.578934+0000", 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0047873259999999997 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.578982+0000", 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7135999999999997e-05 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.579181+0000", 2024-09-15T00:09:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000198879 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.581463+0000", 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022828700000000002 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.583655+0000", 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021912239999999999 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.583683+0000", 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8645000000000001e-05 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.583709+0000", 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6063000000000001e-05 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:1226 3.12 3:4b7b7441:::rbd_data.1129aa143e51.00000000000000ca:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.873141+0000", 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.50884677599999995, 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0079719309999999998, 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1226 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873141+0000", 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873141+0000", 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873141+0000", 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2499999999999995e-07 2024-09-15T00:09:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873145+0000", 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1930000000000002e-06 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873155+0000", 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.022e-05 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873161+0000", 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1510000000000001e-06 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873312+0000", 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150915 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873415+0000", 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010338799999999999 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873562+0000", 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014677700000000001 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.877506+0000", 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039438349999999997 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.881017+0000", 2024-09-15T00:09:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035113509999999998 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.881066+0000", 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8741e-05 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.881113+0000", 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6835000000000001e-05 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4486.0:1233 3.17 3:ee136d0c:::rbd_data.1129aa143e51.000000000000017f:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:06.873202+0000", 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.50878551100000002, 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0078391810000000006, 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4486", 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/788289133", 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1233 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873202+0000", 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873202+0000", 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873203+0000", 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.51e-07 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873205+0000", 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.672e-06 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873212+0000", 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7259999999999996e-06 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873217+0000", 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1819999999999997e-06 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873263+0000", 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6022000000000001e-05 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873330+0000", 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6747000000000003e-05 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 5", 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.873496+0000", 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016626299999999999 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.877529+0000", 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040327549999999998 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.880949+0000", 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034200929999999999 2024-09-15T00:09:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.881000+0000", 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0933999999999997e-05 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:06.881041+0000", 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1236000000000003e-05 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:07.510 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:07.512 INFO:teuthology.orchestra.run.smithi092.stdout:391 punch from 0x20e200 to 0x218200, (0xa000 bytes) 2024-09-15T00:09:07.512 INFO:teuthology.orchestra.run.smithi092.stdout:392 read 0x6acc00 thru 0x6bb99c (0xed9d bytes) 2024-09-15T00:09:07.512 INFO:teuthology.orchestra.run.smithi092.stdout:397 read 0xbbcc00 thru 0xbbf497 (0x2898 bytes) 2024-09-15T00:09:07.512 INFO:teuthology.orchestra.run.smithi092.stdout:398 read 0x322200 thru 0x33195d (0xf75e bytes) 2024-09-15T00:09:07.512 INFO:teuthology.orchestra.run.smithi092.stdout:400 read 0xd34a00 thru 0xd3e9b9 (0x9fba bytes) 2024-09-15T00:09:07.512 INFO:teuthology.orchestra.run.smithi092.stdout:402 clone 4 order 22 su 65536 sc 15 2024-09-15T00:09:07.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:07.513 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:07.509+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:07.612 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:07.610+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:07.660 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:07.660 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:07.660 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:07.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:07.704 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:07.713 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:07.711+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:07.777 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:07.777 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:07.777 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:07.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:07.829 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:07.827+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:07.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:07.837 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:07.855 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:07.855 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:07.872 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:07.872 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:07.889 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:07.889 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:07.896 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:07.896 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:07.905 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:07.906 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:07.906 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:07.923 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:07.923 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:07.923 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:07.921+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:07.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:09:08.021 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:08.021 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:08.022 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:08.019+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:08.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:08.065 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone3 from 0x1543000 (overlap 0x1543000) to 0x1102600 2024-09-15T00:09:08.116 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:08.114+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:08.196 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:08.196 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:08.205 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:08.246 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:08.244+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:08.317 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:08.316+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:08.355 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:08.355 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:08.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:08.418 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:08.417+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:08.470 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:08.470 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:08.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:08.496 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-15T00:09:08.521 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:08.519+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:08.621 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:08.619+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:08.660 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:08.660 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:08.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:08.722 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:08.720+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:08.820 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:08.821 INFO:teuthology.orchestra.run.smithi092.stdout:skipping zero length punch hole 2024-09-15T00:09:08.821 INFO:teuthology.orchestra.run.smithi092.stdout:405 punch from 0x11d3e00 to 0x11d9000, (0x5200 bytes) 2024-09-15T00:09:08.822 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:08.821+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:08.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:08.853 INFO:teuthology.orchestra.run.smithi092.stdout:406 write 0x20ea400 thru 0x20f8015 (0xdc16 bytes) 2024-09-15T00:09:08.854 INFO:teuthology.orchestra.run.smithi092.stdout:407 punch from 0xa91a00 to 0xa9c800, (0xae00 bytes) 2024-09-15T00:09:08.881 INFO:teuthology.orchestra.run.smithi092.stdout:416 write 0xa696600 thru 0xa69e040 (0x7a41 bytes) 2024-09-15T00:09:08.901 INFO:teuthology.orchestra.run.smithi092.stdout:417 write 0xb22a400 thru 0xb22c234 (0x1e35 bytes) 2024-09-15T00:09:08.902 INFO:teuthology.orchestra.run.smithi092.stdout:421 write 0x8c52e00 thru 0x8c56d17 (0x3f18 bytes) 2024-09-15T00:09:08.903 INFO:teuthology.orchestra.run.smithi092.stdout:422 read 0x8d31400 thru 0x8d33525 (0x2126 bytes) 2024-09-15T00:09:08.903 INFO:teuthology.orchestra.run.smithi092.stdout:428 write 0xa9ce400 thru 0xa9daebb (0xcabc bytes) 2024-09-15T00:09:08.903 INFO:teuthology.orchestra.run.smithi092.stdout:429 read 0xe59600 thru 0xe623aa (0x8dab bytes) 2024-09-15T00:09:08.904 INFO:teuthology.orchestra.run.smithi092.stdout:431 write 0x814ac00 thru 0x8156d85 (0xc186 bytes) 2024-09-15T00:09:08.904 INFO:teuthology.orchestra.run.smithi092.stdout:432 write 0x599800 thru 0x5a7c63 (0xe464 bytes) 2024-09-15T00:09:08.914 INFO:teuthology.orchestra.run.smithi092.stdout:433 trunc from 0xb22c400 to 0x3a7b600 2024-09-15T00:09:08.923 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:08.921+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:09:09.004 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:09.004 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:09.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:09.020 INFO:teuthology.orchestra.run.smithi092.stdout:434 write 0x1318200 thru 0x131c4cc (0x42cd bytes) 2024-09-15T00:09:09.042 INFO:teuthology.orchestra.run.smithi092.stdout:436 write 0x7da9600 thru 0x7db2549 (0x8f4a bytes) 2024-09-15T00:09:09.049 INFO:teuthology.orchestra.run.smithi092.stdout:438 trunc from 0x7db2600 to 0xd994a00 2024-09-15T00:09:09.056 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:09.055+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:09.085 INFO:teuthology.orchestra.run.smithi092.stdout:439 trunc from 0xd994a00 to 0x6d1600 2024-09-15T00:09:09.120 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:09.121 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:09.125 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:09.123+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:09.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:09.138 INFO:teuthology.orchestra.run.smithi092.stdout:440 write 0x5b47000 thru 0x5b4cd43 (0x5d44 bytes) 2024-09-15T00:09:09.138 INFO:teuthology.orchestra.run.smithi092.stdout:441 write 0x25d3c00 thru 0x25e218d (0xe58e bytes) 2024-09-15T00:09:09.139 INFO:teuthology.orchestra.run.smithi092.stdout:442 read 0x1d13a00 thru 0x1d1d94e (0x9f4f bytes) 2024-09-15T00:09:09.155 INFO:teuthology.orchestra.run.smithi092.stdout:443 write 0x69d2200 thru 0x69d3ad8 (0x18d9 bytes) 2024-09-15T00:09:09.161 INFO:teuthology.orchestra.run.smithi092.stdout:445 punch from 0x1549a00 to 0x1556800, (0xce00 bytes) 2024-09-15T00:09:09.176 INFO:teuthology.orchestra.run.smithi092.stdout:450 write 0x9f7b200 thru 0x9f7c32a (0x112b bytes) 2024-09-15T00:09:09.191 INFO:teuthology.orchestra.run.smithi092.stdout:452 write 0xc1fb400 thru 0xc1fbb2d (0x72e bytes) 2024-09-15T00:09:09.193 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xdf21000 2024-09-15T00:09:09.195 INFO:teuthology.orchestra.run.smithi092.stdout:460 trunc from 0xc1fbc00 to 0xdf21000 2024-09-15T00:09:09.206 INFO:teuthology.orchestra.run.smithi092.stdout:461 punch from 0xd15b600 to 0xd15dc00, (0x2600 bytes) 2024-09-15T00:09:09.207 INFO:teuthology.orchestra.run.smithi092.stdout:462 read 0x8d51600 thru 0x8d57f27 (0x6928 bytes) 2024-09-15T00:09:09.207 INFO:teuthology.orchestra.run.smithi092.stdout:464 read 0x2e34a00 thru 0x2e42a6b (0xe06c bytes) 2024-09-15T00:09:09.207 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xeb06400 2024-09-15T00:09:09.208 INFO:teuthology.orchestra.run.smithi092.stdout:465 trunc from 0xdf21000 to 0xeb06400 2024-09-15T00:09:09.217 INFO:teuthology.orchestra.run.smithi092.stdout:466 read 0xaa16e00 thru 0xaa19166 (0x2367 bytes) 2024-09-15T00:09:09.226 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:09.224+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:09.229 INFO:teuthology.orchestra.run.smithi092.stdout:469 trunc from 0xeb06400 to 0x96a5a00 2024-09-15T00:09:09.244 INFO:teuthology.orchestra.run.smithi092.stdout:471 read 0x4e80000 thru 0x4e8eba8 (0xeba9 bytes) 2024-09-15T00:09:09.246 INFO:teuthology.orchestra.run.smithi092.stdout:472 write 0x7104600 thru 0x710f902 (0xb303 bytes) 2024-09-15T00:09:09.246 INFO:teuthology.orchestra.run.smithi092.stdout:473 read 0x7296600 thru 0x729e3dd (0x7dde bytes) 2024-09-15T00:09:09.248 INFO:teuthology.orchestra.run.smithi092.stdout:474 trunc from 0x96a5a00 to 0x8098e00 2024-09-15T00:09:09.261 INFO:teuthology.orchestra.run.smithi092.stdout:475 punch from 0x57f200 to 0x58d400, (0xe200 bytes) 2024-09-15T00:09:09.269 INFO:teuthology.orchestra.run.smithi092.stdout:476 punch from 0x3553400 to 0x355fc00, (0xc800 bytes) 2024-09-15T00:09:09.277 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:09.277 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:09.281 INFO:teuthology.orchestra.run.smithi092.stdout:477 write 0xb822200 thru 0xb82bf57 (0x9d58 bytes) 2024-09-15T00:09:09.282 INFO:teuthology.orchestra.run.smithi092.stdout:478 write 0x12c0c00 thru 0x12c1f37 (0x1338 bytes) 2024-09-15T00:09:09.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:09.287 INFO:teuthology.orchestra.run.smithi092.stdout:479 write 0x4a3600 thru 0x4aac61 (0x7662 bytes) 2024-09-15T00:09:09.302 INFO:teuthology.orchestra.run.smithi092.stdout:480 write 0xc488c00 thru 0xc48db1e (0x4f1f bytes) 2024-09-15T00:09:09.313 INFO:teuthology.orchestra.run.smithi092.stdout:481 trunc from 0xc48dc00 to 0x7ecaa00 2024-09-15T00:09:09.327 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:09.325+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:09.340 INFO:teuthology.orchestra.run.smithi092.stdout:482 trunc from 0x7ecaa00 to 0x3697400 2024-09-15T00:09:09.380 INFO:teuthology.orchestra.run.smithi092.stdout:488 trunc from 0x3697400 to 0xe448600 2024-09-15T00:09:09.389 INFO:teuthology.orchestra.run.smithi092.stdout:490 read 0x1bf7c00 thru 0x1bfe8b4 (0x6cb5 bytes) 2024-09-15T00:09:09.396 INFO:teuthology.orchestra.run.smithi092.stdout:492 write 0x580d000 thru 0x580dd5d (0xd5e bytes) 2024-09-15T00:09:09.398 INFO:teuthology.orchestra.run.smithi092.stdout:494 trunc from 0xe448600 to 0xceaf600 2024-09-15T00:09:09.419 INFO:teuthology.orchestra.run.smithi092.stdout:497 trunc from 0xceaf600 to 0x44da000 2024-09-15T00:09:09.427 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:09.426+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:09.429 INFO:teuthology.orchestra.run.smithi092.stdout:499 read 0x1e36a00 thru 0x1e3bbdf (0x51e0 bytes) 2024-09-15T00:09:09.434 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:09.435 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:09.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:09.445 INFO:teuthology.orchestra.run.smithi092.stdout:500 write 0x622d600 thru 0x623bc7f (0xe680 bytes) 2024-09-15T00:09:09.468 INFO:teuthology.orchestra.run.smithi092.stdout:501 write 0xc22fc00 thru 0xc2399be (0x9dbf bytes) 2024-09-15T00:09:09.470 INFO:teuthology.orchestra.run.smithi092.stdout:503 read 0x2f5a600 thru 0x2f6a5f7 (0xfff8 bytes) 2024-09-15T00:09:09.470 INFO:teuthology.orchestra.run.smithi092.stdout:505 read 0x4a14a00 thru 0x4a21c08 (0xd209 bytes) 2024-09-15T00:09:09.471 INFO:teuthology.orchestra.run.smithi092.stdout:511 read 0x17f6600 thru 0x180032e (0x9d2f bytes) 2024-09-15T00:09:09.475 INFO:teuthology.orchestra.run.smithi092.stdout:512 trunc from 0xc239a00 to 0x891b400 2024-09-15T00:09:09.496 INFO:teuthology.orchestra.run.smithi092.stdout:515 read 0x2999c00 thru 0x29a4ac8 (0xaec9 bytes) 2024-09-15T00:09:09.497 INFO:teuthology.orchestra.run.smithi092.stdout:516 read 0x2066800 thru 0x207150a (0xad0b bytes) 2024-09-15T00:09:09.500 INFO:teuthology.orchestra.run.smithi092.stdout:519 trunc from 0x891b400 to 0xa60c200 2024-09-15T00:09:09.509 INFO:teuthology.orchestra.run.smithi092.stdout:520 read 0x9bb1200 thru 0x9bc01d9 (0xefda bytes) 2024-09-15T00:09:09.522 INFO:teuthology.orchestra.run.smithi092.stdout:521 trunc from 0xa60c200 to 0x3c9c00 2024-09-15T00:09:09.529 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:09.527+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:09.541 INFO:teuthology.orchestra.run.smithi092.stdout:523 clone 5 order 23 su 2097152 sc 6 2024-09-15T00:09:09.590 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:09.590 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:09.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:09.629 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:09.628+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:09.700 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:09.701 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:09.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:09.753 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:09.751+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:09.819 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:09.820 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:09.829 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:09.869 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:09.867+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:09.932 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:09.930+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:09.934 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:09.976 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:09.976 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:09.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:10.033 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:10.031+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:10.065 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:10.066 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:10.083 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:10.083 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:10.083 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone4 2024-09-15T00:09:10.103 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:10.103 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:10.121 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:10.121 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:10.133 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:10.131+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:10.137 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:10.138 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:10.142 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:768 3.1f 3:fb2a7c64:::rbd_data.1129aa143e51.0000000000000033:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351183+0000", 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7888751279999999, 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012254846999999999, 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 768 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351183+0000", 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351183+0000", 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351184+0000", 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.08e-07 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351186+0000", 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351192+0000", 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3799999999999999e-06 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351197+0000", 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0250000000000002e-06 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363234+0000", 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01203688 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363323+0000", 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9169e-05 2024-09-15T00:09:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363438+0000", 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011500700000000001 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:770 3.1f 3:fb2a7c64:::rbd_data.1129aa143e51.0000000000000033:5 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351204+0000", 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7888549400000002, 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012615664, 2024-09-15T00:09:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 770 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351204+0000", 2024-09-15T00:09:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351204+0000", 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351204+0000", 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.39e-07 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351206+0000", 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351212+0000", 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9120000000000003e-06 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351217+0000", 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6519999999999997e-06 2024-09-15T00:09:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363470+0000", 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012253429 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363550+0000", 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0277000000000004e-05 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363819+0000", 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000268888 2024-09-15T00:09:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:772 3.1f 3:fb2a7c64:::rbd_data.1129aa143e51.0000000000000033:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351223+0000", 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7888358119999999, 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012778787999999999, 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 772 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351223+0000", 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351223+0000", 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351223+0000", 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.72e-07 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351225+0000", 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2110000000000001e-06 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351231+0000", 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0079999999999999e-06 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351236+0000", 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363854+0000", 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012618076000000001 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363925+0000", 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0976999999999995e-05 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364002+0000", 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6254999999999994e-05 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:817 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351242+0000", 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.788816535, 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012974766, 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 817 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351242+0000", 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351242+0000", 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351243+0000", 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351245+0000", 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2390000000000001e-06 2024-09-15T00:09:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351251+0000", 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0090000000000004e-06 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351256+0000", 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6929999999999998e-06 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364032+0000", 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012776556 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364117+0000", 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4902999999999997e-05 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364217+0000", 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9863000000000003e-05 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:828 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351262+0000", 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7887962580000001, 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013218841, 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 828 2024-09-15T00:09:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351262+0000", 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351262+0000", 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351263+0000", 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351271+0000", 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.96e-06 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351277+0000", 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5490000000000003e-06 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351284+0000", 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364266+0000", 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012981901000000001 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364358+0000", 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2145000000000006e-05 2024-09-15T00:09:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364481+0000", 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000122953 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1270 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355859+0000", 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7841994420000002, 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012140272000000001, 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1270 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355859+0000", 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355859+0000", 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355860+0000", 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355862+0000", 2024-09-15T00:09:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.216e-06 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355869+0000", 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.109e-06 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355875+0000", 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367642+0000", 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01176779 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367728+0000", 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5809000000000005e-05 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367999+0000", 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027126299999999999 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1278 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355882+0000", 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7841768070000001, 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012358408, 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1278 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355882+0000", 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355882+0000", 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355882+0000", 2024-09-15T00:09:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.39e-07 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355884+0000", 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.995e-06 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355891+0000", 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9870000000000002e-06 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355897+0000", 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5099999999999998e-06 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368049+0000", 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012152586999999999 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368136+0000", 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6255999999999995e-05 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368240+0000", 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000104634 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1279 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355903+0000", 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.784155427, 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012602021999999999, 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1279 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355903+0000", 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355903+0000", 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355904+0000", 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355906+0000", 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9240000000000001e-06 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355913+0000", 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.029e-06 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355918+0000", 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6080000000000003e-06 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368287+0000", 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012368580000000001 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368382+0000", 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5455e-05 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368505+0000", 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012301599999999999 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1287 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355925+0000", 2024-09-15T00:09:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7841337670000001, 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012827722999999999, 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1287 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355925+0000", 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355925+0000", 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355925+0000", 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355927+0000", 2024-09-15T00:09:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9530000000000002e-06 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355934+0000", 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5520000000000001e-06 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355939+0000", 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4369999999999998e-06 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368557+0000", 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012617712999999999 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368643+0000", 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6011e-05 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368753+0000", 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109638 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1288 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355946+0000", 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7841125180000001, 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013021272, 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1288 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355946+0000", 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355946+0000", 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355947+0000", 2024-09-15T00:09:10.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3099999999999998e-07 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355949+0000", 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1579999999999999e-06 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355961+0000", 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1901e-05 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355967+0000", 2024-09-15T00:09:10.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9399999999999999e-06 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368787+0000", 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012820249000000001 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368868+0000", 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1384999999999994e-05 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368967+0000", 2024-09-15T00:09:10.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9208000000000003e-05 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1295 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355974+0000", 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7840843849999999, 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013206915, 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.187 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1295 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355974+0000", 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355974+0000", 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355975+0000", 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6800000000000001e-07 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355977+0000", 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1320000000000001e-06 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355984+0000", 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8650000000000003e-06 2024-09-15T00:09:10.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355990+0000", 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8839999999999999e-06 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368999+0000", 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013009121 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369080+0000", 2024-09-15T00:09:10.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1038000000000006e-05 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369181+0000", 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101407 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1296 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 401408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:10.191 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7840585879999997, 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013357545, 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1296 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:10.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356001+0000", 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356003+0000", 2024-09-15T00:09:10.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.486e-06 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356010+0000", 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3459999999999996e-06 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356016+0000", 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4990000000000002e-06 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369219+0000", 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013202894999999999 2024-09-15T00:09:10.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369291+0000", 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2458000000000003e-05 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369358+0000", 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6370000000000003e-05 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1325 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 294912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356044+0000", 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7840143020000001, 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013560946000000001, 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1325 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356044+0000", 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356044+0000", 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356045+0000", 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2300000000000002e-07 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356048+0000", 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5729999999999999e-06 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356057+0000", 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0440000000000001e-06 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356063+0000", 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8619999999999999e-06 2024-09-15T00:09:10.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369423+0000", 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013360276000000001 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369513+0000", 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9617000000000006e-05 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369605+0000", 2024-09-15T00:09:10.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2151000000000006e-05 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1326 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356070+0000", 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7839881470000001, 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013717217, 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1326 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356070+0000", 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356070+0000", 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356071+0000", 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6199999999999998e-07 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356073+0000", 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0640000000000001e-06 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356080+0000", 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7329999999999997e-06 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356085+0000", 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0649999999999998e-06 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369652+0000", 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013567106000000001 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369722+0000", 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9794000000000003e-05 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369788+0000", 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5993000000000003e-05 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1334 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 491520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356091+0000", 2024-09-15T00:09:10.204 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7839671269999999, 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013921717, 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1334 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356091+0000", 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356091+0000", 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356092+0000", 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.27e-07 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356094+0000", 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0530000000000001e-06 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356101+0000", 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6460000000000004e-06 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356106+0000", 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1880000000000001e-06 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369824+0000", 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013718340000000001 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369905+0000", 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0694999999999999e-05 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370013+0000", 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108368 2024-09-15T00:09:10.208 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1335 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356112+0000", 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7839463589999998, 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014060968, 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.209 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1335 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356112+0000", 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356112+0000", 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356124+0000", 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1223000000000001e-05 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356131+0000", 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1860000000000003e-06 2024-09-15T00:09:10.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356138+0000", 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2089999999999999e-06 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370081+0000", 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01394226 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370124+0000", 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3551999999999999e-05 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370173+0000", 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9060000000000001e-05 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1364 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356168+0000", 2024-09-15T00:09:10.213 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.78389087, 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014233545, 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1364 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356168+0000", 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356168+0000", 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356168+0000", 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:10.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.046e-06 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356177+0000", 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5039999999999999e-06 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356182+0000", 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8659999999999997e-06 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370209+0000", 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014027181 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370284+0000", 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.525e-05 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370401+0000", 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011728800000000001 2024-09-15T00:09:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1365 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356189+0000", 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7838699939999998, 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014419042999999999, 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.218 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1365 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356189+0000", 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356189+0000", 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356189+0000", 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4999999999999998e-07 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356191+0000", 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356198+0000", 2024-09-15T00:09:10.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7730000000000002e-06 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356204+0000", 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5890000000000002e-06 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370434+0000", 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014229980999999999 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370508+0000", 2024-09-15T00:09:10.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4011999999999995e-05 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370608+0000", 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100058 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1373 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356210+0000", 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.7838490039999999, 2024-09-15T00:09:10.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0145708, 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1373 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356210+0000", 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356210+0000", 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356210+0000", 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.65e-07 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356212+0000", 2024-09-15T00:09:10.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.858e-06 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356218+0000", 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5710000000000003e-06 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356224+0000", 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370640+0000", 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014416698 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370693+0000", 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2326e-05 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370780+0000", 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7774000000000004e-05 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.226 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1374 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 2.783828642, 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014785247, 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1374 2024-09-15T00:09:10.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.01e-07 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356232+0000", 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356239+0000", 2024-09-15T00:09:10.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4030000000000004e-06 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356244+0000", 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8749999999999999e-06 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370813+0000", 2024-09-15T00:09:10.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014569248999999999 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370913+0000", 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100144 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371015+0000", 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102148 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.232 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.232 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:10.233 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:10.233 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:10.234 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:10.275 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:10.273+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:10.336 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:10.334+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:10.389 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-15T00:09:10.395 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:10.436 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:10.435+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:10.537 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:10.536+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:10.557 INFO:teuthology.orchestra.run.smithi092.stdout:524 read 0x396400 thru 0x39cc33 (0x6834 bytes) 2024-09-15T00:09:10.558 INFO:teuthology.orchestra.run.smithi092.stdout:525 read 0x35ee00 thru 0x3677b2 (0x89b3 bytes) 2024-09-15T00:09:10.559 INFO:teuthology.orchestra.run.smithi092.stdout:527 read 0xe400 thru 0xf93e (0x153f bytes) 2024-09-15T00:09:10.560 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:10.560 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:10.560 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:10.561 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:10.561 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:10.569 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:10.588 INFO:teuthology.orchestra.run.smithi092.stdout:529 write 0x7637600 thru 0x763f32c (0x7d2d bytes) 2024-09-15T00:09:10.588 INFO:teuthology.orchestra.run.smithi092.stdout:530 read 0x2626400 thru 0x262a627 (0x4228 bytes) 2024-09-15T00:09:10.588 INFO:teuthology.orchestra.run.smithi092.stdout:531 punch from 0x20f3600 to 0x20fd800, (0xa200 bytes) 2024-09-15T00:09:10.588 INFO:teuthology.orchestra.run.smithi092.stdout:532 read 0x3cc0800 thru 0x3ccdd82 (0xd583 bytes) 2024-09-15T00:09:10.588 INFO:teuthology.orchestra.run.smithi092.stdout:537 read 0x338c800 thru 0x338d026 (0x827 bytes) 2024-09-15T00:09:10.588 INFO:teuthology.orchestra.run.smithi092.stdout:539 punch from 0x2d85200 to 0x2d8f800, (0xa600 bytes) 2024-09-15T00:09:10.588 INFO:teuthology.orchestra.run.smithi092.stdout:542 write 0x18bd200 thru 0x18c5821 (0x8622 bytes) 2024-09-15T00:09:10.608 INFO:teuthology.orchestra.run.smithi092.stdout:543 write 0x7893a00 thru 0x789fbc8 (0xc1c9 bytes) 2024-09-15T00:09:10.609 INFO:teuthology.orchestra.run.smithi092.stdout:544 read 0x431b600 thru 0x43248ad (0x92ae bytes) 2024-09-15T00:09:10.609 INFO:teuthology.orchestra.run.smithi092.stdout:547 punch from 0x37bea00 to 0x37ccc00, (0xe200 bytes) 2024-09-15T00:09:10.609 INFO:teuthology.orchestra.run.smithi092.stdout:548 read 0x6273800 thru 0x6276db5 (0x35b6 bytes) 2024-09-15T00:09:10.609 INFO:teuthology.orchestra.run.smithi092.stdout:552 punch from 0x5febe00 to 0x5ffb400, (0xf600 bytes) 2024-09-15T00:09:10.609 INFO:teuthology.orchestra.run.smithi092.stdout:553 write 0x5244e00 thru 0x524f3ab (0xa5ac bytes) 2024-09-15T00:09:10.609 INFO:teuthology.orchestra.run.smithi092.stdout:557 write 0x21aba00 thru 0x21b80c5 (0xc6c6 bytes) 2024-09-15T00:09:10.610 INFO:teuthology.orchestra.run.smithi092.stdout:558 read 0x71b8600 thru 0x71bc29f (0x3ca0 bytes) 2024-09-15T00:09:10.610 INFO:teuthology.orchestra.run.smithi092.stdout:561 punch from 0xb4e800 to 0xb5d200, (0xea00 bytes) 2024-09-15T00:09:10.623 INFO:teuthology.orchestra.run.smithi092.stdout:562 write 0xaaf2a00 thru 0xab003ef (0xd9f0 bytes) 2024-09-15T00:09:10.624 INFO:teuthology.orchestra.run.smithi092.stdout:565 punch from 0x1ba3c00 to 0x1ba6c00, (0x3000 bytes) 2024-09-15T00:09:10.625 INFO:teuthology.orchestra.run.smithi092.stdout:566 punch from 0x7189e00 to 0x7194400, (0xa600 bytes) 2024-09-15T00:09:10.628 INFO:teuthology.orchestra.run.smithi092.stdout:567 punch from 0x9165e00 to 0x9172000, (0xc200 bytes) 2024-09-15T00:09:10.632 INFO:teuthology.orchestra.run.smithi092.stdout:568 trunc from 0xab00400 to 0x46d2600 2024-09-15T00:09:10.638 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:10.637+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:10.655 INFO:teuthology.orchestra.run.smithi092.stdout:569 write 0x1eaca00 thru 0x1eb6f2a (0xa52b bytes) 2024-09-15T00:09:10.662 INFO:teuthology.orchestra.run.smithi092.stdout:571 read 0x3446400 thru 0x34545ab (0xe1ac bytes) 2024-09-15T00:09:10.662 INFO:teuthology.orchestra.run.smithi092.stdout:572 punch from 0x922600 to 0x92be00, (0x9800 bytes) 2024-09-15T00:09:10.675 INFO:teuthology.orchestra.run.smithi092.stdout:575 write 0x7d1f800 thru 0x7d2da8b (0xe28c bytes) 2024-09-15T00:09:10.676 INFO:teuthology.orchestra.run.smithi092.stdout:576 write 0x3276c00 thru 0x3285922 (0xed23 bytes) 2024-09-15T00:09:10.676 INFO:teuthology.orchestra.run.smithi092.stdout:577 read 0x51d5800 thru 0x51e1766 (0xbf67 bytes) 2024-09-15T00:09:10.677 INFO:teuthology.orchestra.run.smithi092.stdout:578 trunc from 0x7d2dc00 to 0x88aee00 2024-09-15T00:09:10.688 INFO:teuthology.orchestra.run.smithi092.stdout:579 punch from 0x1efa400 to 0x1f09600, (0xf200 bytes) 2024-09-15T00:09:10.690 INFO:teuthology.orchestra.run.smithi092.stdout:580 read 0x59efe00 thru 0x59f3bba (0x3dbb bytes) 2024-09-15T00:09:10.692 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1237 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 401408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353675+0000", 2024-09-15T00:09:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3358043030000002, 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013309087000000001, 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1237 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353675+0000", 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353675+0000", 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353676+0000", 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.34e-07 2024-09-15T00:09:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353678+0000", 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.99e-06 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353684+0000", 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0909999999999998e-06 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353689+0000", 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8760000000000004e-06 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.366810+0000", 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013120672999999999 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.366889+0000", 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9314999999999995e-05 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.366985+0000", 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5707999999999999e-05 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1238 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353694+0000", 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.335785343, 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013468529, 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1238 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353694+0000", 2024-09-15T00:09:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353694+0000", 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353695+0000", 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353697+0000", 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.764e-06 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353702+0000", 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7649999999999999e-06 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353708+0000", 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1419999999999998e-06 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367014+0000", 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013305895999999999 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367082+0000", 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7415000000000004e-05 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367163+0000", 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1170999999999999e-05 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1239 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353715+0000", 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3357643370000001, 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013586938999999999, 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1239 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353715+0000", 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353715+0000", 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353716+0000", 2024-09-15T00:09:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:09:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353718+0000", 2024-09-15T00:09:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.897e-06 2024-09-15T00:09:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353724+0000", 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8250000000000001e-06 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353728+0000", 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5589999999999999e-06 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367198+0000", 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013470058 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367246+0000", 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8013000000000003e-05 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367302+0000", 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6239999999999997e-05 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1240 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.335745255, 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013711031, 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1240 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:09:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353737+0000", 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.844e-06 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353744+0000", 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0310000000000001e-06 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353748+0000", 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6569999999999996e-06 2024-09-15T00:09:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367332+0000", 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013583556 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367368+0000", 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5796000000000001e-05 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367446+0000", 2024-09-15T00:09:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7792000000000003e-05 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1241 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353754+0000", 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3357256180000001, 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013907709000000001, 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1241 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353754+0000", 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353754+0000", 2024-09-15T00:09:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6100000000000002e-07 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353756+0000", 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.818e-06 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353764+0000", 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.435e-06 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353768+0000", 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6970000000000001e-06 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367483+0000", 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013714835999999999 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367559+0000", 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6106000000000006e-05 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367662+0000", 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102456 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1242 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353774+0000", 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3357054399999999, 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014339533, 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1242 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353774+0000", 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353774+0000", 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353775+0000", 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.39e-07 2024-09-15T00:09:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353777+0000", 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8950000000000001e-06 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353782+0000", 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8479999999999997e-06 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353801+0000", 2024-09-15T00:09:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8209000000000001e-05 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367715+0000", 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013914358 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367811+0000", 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5613999999999996e-05 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368114+0000", 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030327000000000002 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1244 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.335646739, 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014385056, 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1244 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353836+0000", 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5670000000000002e-06 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353845+0000", 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7630000000000006e-06 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353850+0000", 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2850000000000002e-06 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368136+0000", 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014285791000000001 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368174+0000", 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8414999999999997e-05 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368218+0000", 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3791999999999997e-05 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1245 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353858+0000", 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3356215410000001, 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014529564999999999, 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1245 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353858+0000", 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353858+0000", 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353859+0000", 2024-09-15T00:09:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9800000000000004e-07 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353861+0000", 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0059999999999999e-06 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353867+0000", 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9780000000000002e-06 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353871+0000", 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6779999999999999e-06 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368248+0000", 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014376172 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368313+0000", 2024-09-15T00:09:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5597000000000002e-05 2024-09-15T00:09:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368388+0000", 2024-09-15T00:09:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4635999999999995e-05 2024-09-15T00:09:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1264 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3355561950000001, 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014693985, 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1264 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6499999999999999e-07 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353927+0000", 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1489999999999998e-06 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353934+0000", 2024-09-15T00:09:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0299999999999996e-06 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353940+0000", 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5759999999999996e-06 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368429+0000", 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014489007999999999 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368517+0000", 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8008000000000002e-05 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368618+0000", 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010074899999999999 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1265 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3355310309999999, 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01488623, 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1265 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353951+0000", 2024-09-15T00:09:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353958+0000", 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2620000000000004e-06 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353963+0000", 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9819999999999999e-06 2024-09-15T00:09:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368649+0000", 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014686716000000001 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368734+0000", 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4893000000000002e-05 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368835+0000", 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010072 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1271 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3354636759999998, 2024-09-15T00:09:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015034792999999999, 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1271 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354018+0000", 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8059999999999999e-06 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354024+0000", 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8540000000000002e-06 2024-09-15T00:09:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354029+0000", 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6550000000000003e-06 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368868+0000", 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014839395999999999 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368954+0000", 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5636999999999993e-05 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369051+0000", 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7095999999999993e-05 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1272 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354034+0000", 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3354453720000001, 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015239717, 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1272 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354034+0000", 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354034+0000", 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354035+0000", 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354037+0000", 2024-09-15T00:09:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.849e-06 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354042+0000", 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7250000000000002e-06 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354047+0000", 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369095+0000", 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015047955999999999 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369177+0000", 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2212000000000005e-05 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369274+0000", 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6811999999999994e-05 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1280 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354092+0000", 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3353875089999998, 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01540946, 2024-09-15T00:09:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1280 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354092+0000", 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354092+0000", 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354093+0000", 2024-09-15T00:09:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354094+0000", 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.824e-06 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354100+0000", 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8259999999999998e-06 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354105+0000", 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4309999999999996e-06 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369307+0000", 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015202466 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369401+0000", 2024-09-15T00:09:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4327999999999995e-05 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369502+0000", 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100243 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1289 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354154+0000", 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3353254039999998, 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015611507, 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1289 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354154+0000", 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354154+0000", 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354155+0000", 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8500000000000002e-07 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354157+0000", 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.26e-06 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354163+0000", 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8220000000000003e-06 2024-09-15T00:09:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354168+0000", 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8899999999999998e-06 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369559+0000", 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015391103999999999 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369656+0000", 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6877000000000001e-05 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369766+0000", 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110169 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1316 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3352873280000002, 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015835378000000001, 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1316 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354193+0000", 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354195+0000", 2024-09-15T00:09:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.77e-06 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354200+0000", 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6670000000000001e-06 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354205+0000", 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.78e-06 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369836+0000", 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015630498 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369924+0000", 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8707000000000003e-05 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370028+0000", 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010359 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1317 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3352690479999998, 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016038510999999998, 2024-09-15T00:09:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1317 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354219+0000", 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3310000000000004e-06 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354227+0000", 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354232+0000", 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.758e-06 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370061+0000", 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015829006 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370151+0000", 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9575999999999998e-05 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370249+0000", 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8743e-05 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1327 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3351791039999998, 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016224237999999998, 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1327 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354303+0000", 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.277e-06 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354310+0000", 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2450000000000003e-06 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354315+0000", 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0799999999999996e-06 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370290+0000", 2024-09-15T00:09:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015975552000000001 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370413+0000", 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012316399999999999 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370525+0000", 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000111537 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1355 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3351231299999999, 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016460124999999999, 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1355 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354359+0000", 2024-09-15T00:09:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0590000000000001e-06 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354365+0000", 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7769999999999999e-06 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354370+0000", 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370550+0000", 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016180486000000001 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370595+0000", 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5055999999999997e-05 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370817+0000", 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022162199999999999 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1356 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.335104802, 2024-09-15T00:09:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016655262000000001, 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1356 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5600000000000001e-07 2024-09-15T00:09:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354377+0000", 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7710000000000001e-06 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354383+0000", 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8429999999999998e-06 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354388+0000", 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.583e-06 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370848+0000", 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016460174000000001 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370934+0000", 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6431e-05 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371030+0000", 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6003999999999998e-05 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1366 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3350142219999999, 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016710004000000001, 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1366 2024-09-15T00:09:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354468+0000", 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.159e-06 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354474+0000", 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2720000000000003e-06 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354479+0000", 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8420000000000004e-06 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371054+0000", 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016575185999999999 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371110+0000", 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5356999999999999e-05 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371176+0000", 2024-09-15T00:09:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5801000000000002e-05 2024-09-15T00:09:10.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.795 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:10.795 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:10.795 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:10.797 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:10.794+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:10.798 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:10.799 INFO:teuthology.orchestra.run.smithi092.stdout:581 trunc from 0x88aee00 to 0xb64f000 2024-09-15T00:09:10.799 INFO:teuthology.orchestra.run.smithi092.stdout:584 punch from 0xa42da00 to 0xa436600, (0x8c00 bytes) 2024-09-15T00:09:10.799 INFO:teuthology.orchestra.run.smithi092.stdout:586 write 0xbe3dc00 thru 0xbe3e9d0 (0xdd1 bytes) 2024-09-15T00:09:10.799 INFO:teuthology.orchestra.run.smithi092.stdout:587 trunc from 0xbe3ea00 to 0x9a93e00 2024-09-15T00:09:10.799 INFO:teuthology.orchestra.run.smithi092.stdout:590 read 0x3407000 thru 0x3411dbe (0xadbf bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:592 read 0x716c600 thru 0x716c7fa (0x1fb bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:595 write 0x1fabe00 thru 0x1fbaf5a (0xf15b bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:596 read 0x87ee600 thru 0x87fa261 (0xbc62 bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:598 write 0x25f5c00 thru 0x25f77a9 (0x1baa bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:600 read 0x1e83000 thru 0x1e8ac25 (0x7c26 bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:602 write 0xb01d000 thru 0xb02905e (0xc05f bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:603 trunc from 0xb029200 to 0xb884800 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:604 punch from 0xb15d200 to 0xb163200, (0x6000 bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:605 read 0x6769200 thru 0x6771349 (0x814a bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:606 punch from 0x44d6800 to 0x44e5200, (0xea00 bytes) 2024-09-15T00:09:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:611 trunc from 0xb884800 to 0x76c8e00 2024-09-15T00:09:10.805 INFO:teuthology.orchestra.run.smithi092.stdout:612 read 0x420e400 thru 0x421220c (0x3e0d bytes) 2024-09-15T00:09:10.811 INFO:teuthology.orchestra.run.smithi092.stdout:613 read 0x6128400 thru 0x61286fa (0x2fb bytes) 2024-09-15T00:09:10.811 INFO:teuthology.orchestra.run.smithi092.stdout:615 punch from 0x688ea00 to 0x689e200, (0xf800 bytes) 2024-09-15T00:09:10.812 INFO:teuthology.orchestra.run.smithi092.stdout:616 write 0x2118c00 thru 0x2124775 (0xbb76 bytes) 2024-09-15T00:09:10.812 INFO:teuthology.orchestra.run.smithi092.stdout:618 punch from 0x93d200 to 0x943c00, (0x6a00 bytes) 2024-09-15T00:09:10.812 INFO:teuthology.orchestra.run.smithi092.stdout:620 write 0x5951e00 thru 0x595555c (0x375d bytes) 2024-09-15T00:09:10.829 INFO:teuthology.orchestra.run.smithi092.stdout:621 write 0xc1fc000 thru 0xc1fc1f3 (0x1f4 bytes) 2024-09-15T00:09:10.835 INFO:teuthology.orchestra.run.smithi092.stdout:625 punch from 0xccc600 to 0xccfc00, (0x3600 bytes) 2024-09-15T00:09:10.835 INFO:teuthology.orchestra.run.smithi092.stdout:626 read 0x45b7e00 thru 0x45c460f (0xc810 bytes) 2024-09-15T00:09:10.838 INFO:teuthology.orchestra.run.smithi092.stdout:628 trunc from 0xc1fc200 to 0xab48200 2024-09-15T00:09:10.853 INFO:teuthology.orchestra.run.smithi092.stdout:630 write 0x1c17600 thru 0x1c27490 (0xfe91 bytes) 2024-09-15T00:09:10.854 INFO:teuthology.orchestra.run.smithi092.stdout:633 read 0x8bd2e00 thru 0x8bd389d (0xa9e bytes) 2024-09-15T00:09:10.854 INFO:teuthology.orchestra.run.smithi092.stdout:634 punch from 0x489ac00 to 0x48a9e00, (0xf200 bytes) 2024-09-15T00:09:10.855 INFO:teuthology.orchestra.run.smithi092.stdout:635 punch from 0x3d30400 to 0x3d37e00, (0x7a00 bytes) 2024-09-15T00:09:10.855 INFO:teuthology.orchestra.run.smithi092.stdout:636 punch from 0x18d6200 to 0x18d7e00, (0x1c00 bytes) 2024-09-15T00:09:10.869 INFO:teuthology.orchestra.run.smithi092.stdout:637 write 0xd3c1e00 thru 0xd3d0f6f (0xf170 bytes) 2024-09-15T00:09:10.870 INFO:teuthology.orchestra.run.smithi092.stdout:641 read 0x3f5cc00 thru 0x3f69ff3 (0xd3f4 bytes) 2024-09-15T00:09:10.871 INFO:teuthology.orchestra.run.smithi092.stdout:644 read 0x346fa00 thru 0x347c9db (0xcfdc bytes) 2024-09-15T00:09:10.871 INFO:teuthology.orchestra.run.smithi092.stdout:646 read 0xa9cee00 thru 0xa9cfeaf (0x10b0 bytes) 2024-09-15T00:09:10.871 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xec0da00 2024-09-15T00:09:10.873 INFO:teuthology.orchestra.run.smithi092.stdout:650 trunc from 0xd3d1000 to 0xec0da00 2024-09-15T00:09:10.884 INFO:teuthology.orchestra.run.smithi092.stdout:651 write 0xc433a00 thru 0xc43aad0 (0x70d1 bytes) 2024-09-15T00:09:10.885 INFO:teuthology.orchestra.run.smithi092.stdout:652 punch from 0xdce8800 to 0xdcee200, (0x5a00 bytes) 2024-09-15T00:09:10.887 INFO:teuthology.orchestra.run.smithi092.stdout:657 read 0x86d4400 thru 0x86df04e (0xac4f bytes) 2024-09-15T00:09:10.887 INFO:teuthology.orchestra.run.smithi092.stdout:659 read 0x9da4e00 thru 0x9da882b (0x3a2c bytes) 2024-09-15T00:09:10.887 INFO:teuthology.orchestra.run.smithi092.stdout:660 read 0x404a400 thru 0x404c2d4 (0x1ed5 bytes) 2024-09-15T00:09:10.888 INFO:teuthology.orchestra.run.smithi092.stdout:663 punch from 0x8257800 to 0x825d600, (0x5e00 bytes) 2024-09-15T00:09:10.893 INFO:teuthology.orchestra.run.smithi092.stdout:664 trunc from 0xec0da00 to 0xa416200 2024-09-15T00:09:10.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:10.895+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:10.916 INFO:teuthology.orchestra.run.smithi092.stdout:666 trunc from 0xa416200 to 0xebfd600 2024-09-15T00:09:10.918 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:10.918 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:10.918 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:10.918 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:10.925 INFO:teuthology.orchestra.run.smithi092.stdout:668 read 0xb94f600 thru 0xb956fb6 (0x79b7 bytes) 2024-09-15T00:09:10.927 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:10.932 INFO:teuthology.orchestra.run.smithi092.stdout:669 read 0x6029600 thru 0x6038bc5 (0xf5c6 bytes) 2024-09-15T00:09:10.932 INFO:teuthology.orchestra.run.smithi092.stdout:670 punch from 0x3d52200 to 0x3d56200, (0x4000 bytes) 2024-09-15T00:09:10.932 INFO:teuthology.orchestra.run.smithi092.stdout:671 write 0x98f2400 thru 0x98fa611 (0x8212 bytes) 2024-09-15T00:09:10.933 INFO:teuthology.orchestra.run.smithi092.stdout:672 read 0x8212000 thru 0x8216b5a (0x4b5b bytes) 2024-09-15T00:09:10.933 INFO:teuthology.orchestra.run.smithi092.stdout:673 read 0x228a400 thru 0x229908f (0xec90 bytes) 2024-09-15T00:09:10.933 INFO:teuthology.orchestra.run.smithi092.stdout:675 read 0xdfea400 thru 0xdff0655 (0x6256 bytes) 2024-09-15T00:09:10.934 INFO:teuthology.orchestra.run.smithi092.stdout:676 read 0x4986e00 thru 0x498cc7c (0x5e7d bytes) 2024-09-15T00:09:10.934 INFO:teuthology.orchestra.run.smithi092.stdout:679 punch from 0x85a6200 to 0x85ab200, (0x5000 bytes) 2024-09-15T00:09:10.934 INFO:teuthology.orchestra.run.smithi092.stdout:680 read 0x1ec7400 thru 0x1ecbacf (0x46d0 bytes) 2024-09-15T00:09:10.942 INFO:teuthology.orchestra.run.smithi092.stdout:681 trunc from 0xebfd600 to 0x87b4600 2024-09-15T00:09:10.983 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-15T00:09:10.983 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:09:10.983 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T00:09:10.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:09:11.000 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:09:11.011 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:11.009+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:11.077 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:11.077 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:11.077 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:11.077 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:11.077 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:11.086 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:11.128 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:11.126+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:11.199 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:11.197+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:11.206 INFO:teuthology.orchestra.run.smithi092.stdout:683 read 0x5d39c00 thru 0x5d40476 (0x6877 bytes) 2024-09-15T00:09:11.207 INFO:teuthology.orchestra.run.smithi092.stdout:684 read 0x2cd6400 thru 0x2cdb8f8 (0x54f9 bytes) 2024-09-15T00:09:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:686 read 0x63ce200 thru 0x63d692b (0x872c bytes) 2024-09-15T00:09:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:690 punch from 0x4d30a00 to 0x4d3a000, (0x9600 bytes) 2024-09-15T00:09:11.208 INFO:teuthology.orchestra.run.smithi092.stdout:692 write 0x26d3000 thru 0x26da259 (0x725a bytes) 2024-09-15T00:09:11.226 INFO:teuthology.orchestra.run.smithi092.stdout:693 write 0x8cb4c00 thru 0x8cbe258 (0x9659 bytes) 2024-09-15T00:09:11.226 INFO:teuthology.orchestra.run.smithi092.stdout:700 punch from 0x251ec00 to 0x252cc00, (0xe000 bytes) 2024-09-15T00:09:11.227 INFO:teuthology.orchestra.run.smithi092.stdout:701 punch from 0x7ceb400 to 0x7cf4800, (0x9400 bytes) 2024-09-15T00:09:11.227 INFO:teuthology.orchestra.run.smithi092.stdout:702 read 0x4e91600 thru 0x4e99cc6 (0x86c7 bytes) 2024-09-15T00:09:11.228 INFO:teuthology.orchestra.run.smithi092.stdout:703 clone 6 order 22 su 1048576 sc 6 2024-09-15T00:09:11.243 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:11.243 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:11.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:11.243 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:11.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.243 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1227 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 491520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.243 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.352925+0000", 2024-09-15T00:09:11.243 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8868340469999998, 2024-09-15T00:09:11.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.026948511000000001, 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1227 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352925+0000", 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352925+0000", 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352926+0000", 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7299999999999996e-07 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352929+0000", 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5170000000000002e-06 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352938+0000", 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9450000000000005e-06 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352944+0000", 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.959e-06 2024-09-15T00:09:11.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379759+0000", 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.026814560000000001 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379808+0000", 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9107999999999998e-05 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379874+0000", 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5848999999999999e-05 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1260 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2023424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.886807734, 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027065815, 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1260 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352960+0000", 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9559999999999997e-06 2024-09-15T00:09:11.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352968+0000", 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5939999999999996e-06 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352973+0000", 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8409999999999999e-06 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379911+0000", 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.026938004000000002 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379958+0000", 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7607e-05 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380018+0000", 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9438999999999998e-05 2024-09-15T00:09:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1269 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8867745810000001, 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027165930000000001, 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.253 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1269 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9100000000000004e-07 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352988+0000", 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4779999999999998e-06 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353002+0000", 2024-09-15T00:09:11.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3699999999999999e-05 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353007+0000", 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5310000000000001e-06 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380046+0000", 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027038801000000001 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380090+0000", 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380151+0000", 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1210000000000005e-05 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1277 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.88674352, 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027261463999999999, 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1277 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:11.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353026+0000", 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3379999999999994e-06 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353034+0000", 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8790000000000003e-06 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353039+0000", 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4489999999999998e-06 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380180+0000", 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.02714068 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380226+0000", 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5776999999999999e-05 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380277+0000", 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1922000000000002e-05 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1285 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:11.262 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8867027759999999, 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027368669000000002, 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1285 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1399999999999997e-07 2024-09-15T00:09:11.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353060+0000", 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4310000000000001e-06 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353067+0000", 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.746e-06 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353073+0000", 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.541e-06 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380305+0000", 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0272325 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380349+0000", 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3995000000000001e-05 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380425+0000", 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5941999999999999e-05 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1286 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.886678861, 2024-09-15T00:09:11.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027475013999999999, 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1286 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353087+0000", 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1299999999999998e-06 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353089+0000", 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.587e-06 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353096+0000", 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1409999999999999e-06 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353102+0000", 2024-09-15T00:09:11.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380455+0000", 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027352936000000001 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380501+0000", 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6236999999999998e-05 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380556+0000", 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4287999999999999e-05 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1292 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8866456540000001, 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027570794999999999, 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1292 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4700000000000001e-07 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353117+0000", 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.295e-06 2024-09-15T00:09:11.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353130+0000", 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2738999999999999e-05 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353141+0000", 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1885000000000001e-05 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380585+0000", 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027443263999999998 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380628+0000", 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3115999999999999e-05 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380685+0000", 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6848999999999997e-05 2024-09-15T00:09:11.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1294 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.886609424, 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027661577, 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.277 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1294 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353151+0000", 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353153+0000", 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353160+0000", 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353165+0000", 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0340000000000004e-06 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380714+0000", 2024-09-15T00:09:11.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027549081 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380758+0000", 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4323000000000002e-05 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380812+0000", 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3414000000000001e-05 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1314 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8863535430000002, 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027532068, 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1314 2024-09-15T00:09:11.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353407+0000", 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6300000000000004e-07 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353411+0000", 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0099999999999997e-06 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353421+0000", 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0309000000000001e-05 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353428+0000", 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7750000000000004e-06 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380839+0000", 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027410994000000001 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380883+0000", 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4079000000000003e-05 2024-09-15T00:09:11.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380938+0000", 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5138e-05 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1320 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.886320209, 2024-09-15T00:09:11.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027624342999999999, 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1320 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353440+0000", 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0399999999999996e-07 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353442+0000", 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3800000000000001e-06 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353449+0000", 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3250000000000002e-06 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353475+0000", 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.584e-05 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380968+0000", 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027492249999999999 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381010+0000", 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2228999999999998e-05 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381064+0000", 2024-09-15T00:09:11.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3814999999999999e-05 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1321 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8862733560000002, 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027730890000000001, 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1321 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353487+0000", 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6499999999999999e-07 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353489+0000", 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7240000000000001e-06 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353504+0000", 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4636e-05 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353510+0000", 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7060000000000001e-06 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381091+0000", 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027581668 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381158+0000", 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6229000000000006e-05 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381217+0000", 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9361999999999998e-05 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.295 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1324 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8860043819999999, 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027587508, 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1324 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353756+0000", 2024-09-15T00:09:11.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6800000000000009e-07 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353760+0000", 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1099999999999996e-06 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353772+0000", 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1464000000000001e-05 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353779+0000", 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3470000000000001e-06 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381246+0000", 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027467155 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381287+0000", 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0553999999999998e-05 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381343+0000", 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5909999999999998e-05 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1332 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.885969303, 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027695646000000001, 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1332 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353791+0000", 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.58e-07 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353793+0000", 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3350000000000001e-06 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353800+0000", 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2599999999999999e-06 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353806+0000", 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6470000000000003e-06 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381370+0000", 2024-09-15T00:09:11.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027564298000000001 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381429+0000", 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9302999999999998e-05 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381486+0000", 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6344999999999997e-05 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.304 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1333 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8859440030000001, 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027796923000000001, 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1333 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353816+0000", 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353818+0000", 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.137e-06 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353824+0000", 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3439999999999997e-06 2024-09-15T00:09:11.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353829+0000", 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381516+0000", 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027686319000000001 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381558+0000", 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1927e-05 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381612+0000", 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4778999999999998e-05 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.309 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1353 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2039808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.885885751, 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027865687, 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1353 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.99e-07 2024-09-15T00:09:11.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353879+0000", 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4089999999999996e-06 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353887+0000", 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1470000000000001e-06 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353893+0000", 2024-09-15T00:09:11.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6840000000000001e-06 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381640+0000", 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027747720999999999 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381683+0000", 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2122000000000001e-05 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381739+0000", 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6904999999999998e-05 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1359 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8856831650000001, 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.02778951, 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1359 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354077+0000", 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7000000000000003e-07 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354090+0000", 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3158e-05 2024-09-15T00:09:11.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354102+0000", 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1572e-05 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354115+0000", 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3297e-05 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381769+0000", 2024-09-15T00:09:11.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027653499000000002 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381811+0000", 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2348000000000002e-05 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381866+0000", 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4765999999999997e-05 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.318 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1360 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8856328100000002, 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027863138999999999, 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1360 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354130+0000", 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.542e-06 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354139+0000", 2024-09-15T00:09:11.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.85e-06 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354144+0000", 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5550000000000002e-06 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381895+0000", 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027750654 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381937+0000", 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2664999999999999e-05 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381990+0000", 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1363 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8855945900000002, 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027949321999999999, 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1363 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354166+0000", 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7199999999999998e-07 2024-09-15T00:09:11.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354169+0000", 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.207e-06 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354178+0000", 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.738e-06 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354184+0000", 2024-09-15T00:09:11.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1040000000000003e-06 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382019+0000", 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027835069 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382059+0000", 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9988999999999999e-05 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382114+0000", 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5543e-05 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1370 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8855670880000002, 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028036106000000002, 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1370 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354193+0000", 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354195+0000", 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1090000000000001e-06 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354210+0000", 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5423000000000001e-05 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354217+0000", 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2269999999999998e-06 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382138+0000", 2024-09-15T00:09:11.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027921069999999999 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382177+0000", 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9412999999999997e-05 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382228+0000", 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1437999999999998e-05 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.332 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1372 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8855312250000003, 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028122709999999999, 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1372 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:11.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354229+0000", 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8599999999999998e-07 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354232+0000", 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.757e-06 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:11.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354239+0000", 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3710000000000002e-06 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354245+0000", 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6300000000000003e-06 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382259+0000", 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028014292999999999 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382301+0000", 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2354999999999997e-05 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382351+0000", 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9617999999999998e-05 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.337 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.338 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:11.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:11.340 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 30, 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "4f532d31-28c1-4236-a74e-6392960baf2b", 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2024-09-15T00:08:32.007191+0000", 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2024-09-15T00:09:09.973269+0000", 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2024-09-15T00:08:42.794810+0000", 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2024-09-15T00:08:33.890963+0000", 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2024-09-15T00:09:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 3, 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 6, 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2024-09-15T00:09:11.342 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "squid", 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:42.896076+0000", 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:11.344 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "14", 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:11.346 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:11.347 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.348 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 6.059999942779541, 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 6.059999942779541, 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.33000001311302185, 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.349 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 2, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "rbd", 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:49.173060+0000", 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:11.350 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 8, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 8, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 8, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 8, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 8, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:11.351 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "18", 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 18, 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:11.352 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:11.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:11.354 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:11.355 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "pool_client.0", 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:58.879058+0000", 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:11.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2024-09-15T00:09:11.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "30", 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 7, 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 30, 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:11.359 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:11.360 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:11.361 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:11.362 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "56600dc2-b95d-47a7-b2b1-5ddc54c08341", 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:11.363 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2024-09-15T00:09:11.364 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:11.365 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.366 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/1440638768", 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/1440638768", 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6823/1440638768", 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6821/1440638768", 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "ea8f0155-8472-4e87-89fd-7716d05065d0", 2024-09-15T00:09:11.368 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:11.371 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/3601510193", 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/3601510193", 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/3601510193", 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/3601510193", 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "357d2d59-66f0-47b8-87c8-4acdfe62a07d", 2024-09-15T00:09:11.374 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 26, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-15T00:09:11.375 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/3222759256", 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/3222759256", 2024-09-15T00:09:11.379 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/3222759256", 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/3222759256", 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "22208f56-d8aa-4a56-a42c-5a990962b860", 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:11.380 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6809", 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6811", 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6813", 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6816", 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6822", 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6823", 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6818", 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6821", 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6811/675428777", 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6816/675428777", 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6823/675428777", 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6821/675428777", 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:11.385 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "a974629e-9ef0-40a3-94bd-382e0bfa8bb6", 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:11.386 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6808", 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6810", 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6812", 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6814", 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.388 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6819", 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6820", 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.389 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6815", 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6817", 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6810/3736884138", 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6814/3736884138", 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6820/3736884138", 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6817/3736884138", 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "06b73c2d-466a-4c4e-89fa-31d6eeb1af1d", 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:11.391 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6800", 2024-09-15T00:09:11.392 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6801", 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.393 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6802", 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6803", 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6806", 2024-09-15T00:09:11.394 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6807", 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:11.395 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6804", 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6805", 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6801/2235671634", 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6803/2235671634", 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6807/2235671634", 2024-09-15T00:09:11.396 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6805/2235671634", 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:11.397 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.730107+0000", 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.365297+0000", 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.868500+0000", 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:11.399 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.788781+0000", 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:11.400 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.892143+0000", 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.990037+0000", 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.401 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.1e", 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 3, 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 4 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2024-09-15T00:09:11.402 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2024-09-15T00:09:11.403 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:11.404 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:11.405 INFO:tasks.thrashosds.thrasher:j is {'epoch': 30, 'fsid': '4f532d31-28c1-4236-a74e-6392960baf2b', 'created': '2024-09-15T00:08:32.007191+0000', 'modified': '2024-09-15T00:09:09.973269+0000', 'last_up_change': '2024-09-15T00:08:42.794810+0000', 'last_in_change': '2024-09-15T00:08:33.890963+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:08:42.896076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:08:49.173060+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:08:58.879058+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 7, 'snap_epoch': 30, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '56600dc2-b95d-47a7-b2b1-5ddc54c08341', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 1440638768}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 1440638768}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 1440638768}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 1440638768}]}, 'public_addr': '172.21.15.27:6817/1440638768', 'cluster_addr': '172.21.15.27:6819/1440638768', 'heartbeat_back_addr': '172.21.15.27:6823/1440638768', 'heartbeat_front_addr': '172.21.15.27:6821/1440638768', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ea8f0155-8472-4e87-89fd-7716d05065d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 3601510193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 3601510193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 3601510193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 3601510193}]}, 'public_addr': '172.21.15.27:6809/3601510193', 'cluster_addr': '172.21.15.27:6811/3601510193', 'heartbeat_back_addr': '172.21.15.27:6815/3601510193', 'heartbeat_front_addr': '172.21.15.27:6813/3601510193', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '357d2d59-66f0-47b8-87c8-4acdfe62a07d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 3222759256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 3222759256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 3222759256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 3222759256}]}, 'public_addr': '172.21.15.27:6801/3222759256', 'cluster_addr': '172.21.15.27:6803/3222759256', 'heartbeat_back_addr': '172.21.15.27:6807/3222759256', 'heartbeat_front_addr': '172.21.15.27:6805/3222759256', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '22208f56-d8aa-4a56-a42c-5a990962b860', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6809', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 675428777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6813', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6816', 'nonce': 675428777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 675428777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 675428777}]}, 'public_addr': '172.21.15.79:6811/675428777', 'cluster_addr': '172.21.15.79:6816/675428777', 'heartbeat_back_addr': '172.21.15.79:6823/675428777', 'heartbeat_front_addr': '172.21.15.79:6821/675428777', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a974629e-9ef0-40a3-94bd-382e0bfa8bb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6810', 'nonce': 3736884138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6814', 'nonce': 3736884138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6819', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6820', 'nonce': 3736884138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6815', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 3736884138}]}, 'public_addr': '172.21.15.79:6810/3736884138', 'cluster_addr': '172.21.15.79:6814/3736884138', 'heartbeat_back_addr': '172.21.15.79:6820/3736884138', 'heartbeat_front_addr': '172.21.15.79:6817/3736884138', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '06b73c2d-466a-4c4e-89fa-31d6eeb1af1d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 2235671634}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2235671634}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2235671634}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2235671634}]}, 'public_addr': '172.21.15.79:6801/2235671634', 'cluster_addr': '172.21.15.79:6803/2235671634', 'heartbeat_back_addr': '172.21.15.79:6807/2235671634', 'heartbeat_front_addr': '172.21.15.79:6805/2235671634', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.730107+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.365297+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.868500+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.788781+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.892143+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.990037+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1e', 'mappings': [{'from': 3, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:09:11.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:09:11.406 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:11.338+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:11.449 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:11.448+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:11.509 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:11.510 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:11.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:11.541 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:11.538+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:11.641 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:11.639+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:11.660 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:11.661 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:11.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:11.695 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:09:11.696 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:09:11.706 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-15T00:09:10.857136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42474025,"num_objects":154,"num_object_clones":41,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":39,"num_read":10395,"num_read_kb":30485,"num_write":1089,"num_write_kb":12403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":957,"ondisk_log_size":957,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":186656,"kb_used_data":3680,"kb_used_omap":9,"kb_used_meta":182838,"kb_avail":566044384,"statfs":{"total":579820584960,"available":579629449216,"internally_reserved":0,"allocated":3768320,"data_stored":12038296,"data_compressed":356655,"data_compressed_allocated":999424,"data_compressed_original":10608640,"omap_allocated":9568,"internal_metadata":187226784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41883573,"num_objects":145,"num_object_clones":41,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":39,"num_read":10215,"num_read_kb":30334,"num_write":922,"num_write_kb":9979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.068662"},"pg_stats":[{"pgid":"3.1f","version":"27'20","reported_seq":257,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.370744+0000","last_change":"2024-09-15T00:09:01.970297+0000","last_active":"2024-09-15T00:09:07.370744+0000","last_peered":"2024-09-15T00:09:07.370744+0000","last_clean":"2024-09-15T00:09:07.370744+0000","last_became_active":"2024-09-15T00:08:59.945793+0000","last_became_peered":"2024-09-15T00:08:59.945793+0000","last_unstale":"2024-09-15T00:09:07.370744+0000","last_undegraded":"2024-09-15T00:09:07.370744+0000","last_fullsized":"2024-09-15T00:09:07.370744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:05:00.587189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075174400000000004,"stat_sum":{"num_bytes":2005520,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":217,"num_read_kb":1331,"num_write":18,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'8","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.439184+0000","last_change":"2024-09-15T00:09:06.797902+0000","last_active":"2024-09-15T00:09:07.439184+0000","last_peered":"2024-09-15T00:09:07.439184+0000","last_clean":"2024-09-15T00:09:07.439184+0000","last_became_active":"2024-09-15T00:09:06.795735+0000","last_became_peered":"2024-09-15T00:09:06.795735+0000","last_unstale":"2024-09-15T00:09:07.439184+0000","last_undegraded":"2024-09-15T00:09:07.439184+0000","last_fullsized":"2024-09-15T00:09:07.439184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:47.398220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031880599999999998,"stat_sum":{"num_bytes":1179648,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":9,"num_read_kb":32,"num_write":9,"num_write_kb":1071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'36","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.427691+0000","last_change":"2024-09-15T00:09:01.965351+0000","last_active":"2024-09-15T00:09:07.427691+0000","last_peered":"2024-09-15T00:09:07.427691+0000","last_clean":"2024-09-15T00:09:07.427691+0000","last_became_active":"2024-09-15T00:08:59.947317+0000","last_became_peered":"2024-09-15T00:08:59.947317+0000","last_unstale":"2024-09-15T00:09:07.427691+0000","last_undegraded":"2024-09-15T00:09:07.427691+0000","last_fullsized":"2024-09-15T00:09:07.427691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:26:13.766490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001450004,"stat_sum":{"num_bytes":548906,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":237,"num_read_kb":224,"num_write":60,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"27'120","reported_seq":477,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.414610+0000","last_change":"2024-09-15T00:09:01.964719+0000","last_active":"2024-09-15T00:09:07.414610+0000","last_peered":"2024-09-15T00:09:07.414610+0000","last_clean":"2024-09-15T00:09:07.414610+0000","last_became_active":"2024-09-15T00:08:59.945681+0000","last_became_peered":"2024-09-15T00:08:59.945681+0000","last_unstale":"2024-09-15T00:09:07.414610+0000","last_undegraded":"2024-09-15T00:09:07.414610+0000","last_fullsized":"2024-09-15T00:09:07.414610+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:25:44.667751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090320299999999999,"stat_sum":{"num_bytes":2314277,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":841,"num_read_kb":2586,"num_write":189,"num_write_kb":379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"27'12","reported_seq":641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.440581+0000","last_change":"2024-09-15T00:09:01.969947+0000","last_active":"2024-09-15T00:09:07.440581+0000","last_peered":"2024-09-15T00:09:07.440581+0000","last_clean":"2024-09-15T00:09:07.440581+0000","last_became_active":"2024-09-15T00:08:59.945617+0000","last_became_peered":"2024-09-15T00:08:59.945617+0000","last_unstale":"2024-09-15T00:09:07.440581+0000","last_undegraded":"2024-09-15T00:09:07.440581+0000","last_fullsized":"2024-09-15T00:09:07.440581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:47:35.823420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048451499999999998,"stat_sum":{"num_bytes":2473984,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":607,"num_read_kb":4457,"num_write":11,"num_write_kb":913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"27'78","reported_seq":1592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.441806+0000","last_change":"2024-09-15T00:09:01.961084+0000","last_active":"2024-09-15T00:09:07.441806+0000","last_peered":"2024-09-15T00:09:07.441806+0000","last_clean":"2024-09-15T00:09:07.441806+0000","last_became_active":"2024-09-15T00:08:59.946828+0000","last_became_peered":"2024-09-15T00:08:59.946828+0000","last_unstale":"2024-09-15T00:09:07.441806+0000","last_undegraded":"2024-09-15T00:09:07.441806+0000","last_fullsized":"2024-09-15T00:09:07.441806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:16:43.043348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085056300000000004,"stat_sum":{"num_bytes":2115127,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2284,"num_read_kb":4512,"num_write":45,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"27'38","reported_seq":480,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.405677+0000","last_change":"2024-09-15T00:09:01.958081+0000","last_active":"2024-09-15T00:09:07.405677+0000","last_peered":"2024-09-15T00:09:07.405677+0000","last_clean":"2024-09-15T00:09:07.405677+0000","last_became_active":"2024-09-15T00:08:59.942052+0000","last_became_peered":"2024-09-15T00:08:59.942052+0000","last_unstale":"2024-09-15T00:09:07.405677+0000","last_undegraded":"2024-09-15T00:09:07.405677+0000","last_fullsized":"2024-09-15T00:09:07.405677+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:31:09.693248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055705900000000005,"stat_sum":{"num_bytes":566784,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":699,"num_read_kb":691,"num_write":28,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"27'24","reported_seq":329,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.439569+0000","last_change":"2024-09-15T00:09:01.978690+0000","last_active":"2024-09-15T00:09:07.439569+0000","last_peered":"2024-09-15T00:09:07.439569+0000","last_clean":"2024-09-15T00:09:07.439569+0000","last_became_active":"2024-09-15T00:08:59.941686+0000","last_became_peered":"2024-09-15T00:08:59.941686+0000","last_unstale":"2024-09-15T00:09:07.439569+0000","last_undegraded":"2024-09-15T00:09:07.439569+0000","last_fullsized":"2024-09-15T00:09:07.439569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:32:59.325908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000308933,"stat_sum":{"num_bytes":4136960,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":284,"num_read_kb":1732,"num_write":25,"num_write_kb":1219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"27'7","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.439403+0000","last_change":"2024-09-15T00:09:01.970454+0000","last_active":"2024-09-15T00:09:07.439403+0000","last_peered":"2024-09-15T00:09:07.439403+0000","last_clean":"2024-09-15T00:09:07.439403+0000","last_became_active":"2024-09-15T00:08:59.946139+0000","last_became_peered":"2024-09-15T00:08:59.946139+0000","last_unstale":"2024-09-15T00:09:07.439403+0000","last_undegraded":"2024-09-15T00:09:07.439403+0000","last_fullsized":"2024-09-15T00:09:07.439403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:33:57.916670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084960000000000005,"stat_sum":{"num_bytes":1068032,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":25,"num_read_kb":167,"num_write":8,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"27'16","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.342425+0000","last_change":"2024-09-15T00:09:01.965029+0000","last_active":"2024-09-15T00:09:07.342425+0000","last_peered":"2024-09-15T00:09:07.342425+0000","last_clean":"2024-09-15T00:09:07.342425+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:09:07.342425+0000","last_undegraded":"2024-09-15T00:09:07.342425+0000","last_fullsized":"2024-09-15T00:09:07.342425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:19:51.242606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062487,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":473,"num_write":16,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'8","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.147077+0000","last_change":"2024-09-15T00:09:01.953424+0000","last_active":"2024-09-15T00:09:07.147077+0000","last_peered":"2024-09-15T00:09:07.147077+0000","last_clean":"2024-09-15T00:09:07.147077+0000","last_became_active":"2024-09-15T00:08:59.941565+0000","last_became_peered":"2024-09-15T00:08:59.941565+0000","last_unstale":"2024-09-15T00:09:07.147077+0000","last_undegraded":"2024-09-15T00:09:07.147077+0000","last_fullsized":"2024-09-15T00:09:07.147077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:38:09.983706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044338899999999998,"stat_sum":{"num_bytes":568832,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":27,"num_read_kb":216,"num_write":5,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"26'13","reported_seq":173,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.373311+0000","last_change":"2024-09-15T00:09:01.953831+0000","last_active":"2024-09-15T00:09:07.373311+0000","last_peered":"2024-09-15T00:09:07.373311+0000","last_clean":"2024-09-15T00:09:07.373311+0000","last_became_active":"2024-09-15T00:08:59.942776+0000","last_became_peered":"2024-09-15T00:08:59.942776+0000","last_unstale":"2024-09-15T00:09:07.373311+0000","last_undegraded":"2024-09-15T00:09:07.373311+0000","last_fullsized":"2024-09-15T00:09:07.373311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:39.575888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079073999999999995,"stat_sum":{"num_bytes":606208,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":592,"num_write":19,"num_write_kb":411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.790254+0000","last_change":"2024-09-15T00:08:52.835701+0000","last_active":"2024-09-15T00:09:06.790254+0000","last_peered":"2024-09-15T00:09:06.790254+0000","last_clean":"2024-09-15T00:09:06.790254+0000","last_became_active":"2024-09-15T00:08:50.969842+0000","last_became_peered":"2024-09-15T00:08:50.969842+0000","last_unstale":"2024-09-15T00:09:06.790254+0000","last_undegraded":"2024-09-15T00:09:06.790254+0000","last_fullsized":"2024-09-15T00:09:06.790254+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:00.926947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"27'25","reported_seq":229,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.405126+0000","last_change":"2024-09-15T00:09:01.970380+0000","last_active":"2024-09-15T00:09:07.405126+0000","last_peered":"2024-09-15T00:09:07.405126+0000","last_clean":"2024-09-15T00:09:07.405126+0000","last_became_active":"2024-09-15T00:08:59.946388+0000","last_became_peered":"2024-09-15T00:08:59.946388+0000","last_unstale":"2024-09-15T00:09:07.405126+0000","last_undegraded":"2024-09-15T00:09:07.405126+0000","last_fullsized":"2024-09-15T00:09:07.405126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:15.358202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083591999999999998,"stat_sum":{"num_bytes":4988928,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":181,"num_read_kb":1083,"num_write":22,"num_write_kb":373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.792043+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:09:06.792043+0000","last_peered":"2024-09-15T00:09:06.792043+0000","last_clean":"2024-09-15T00:09:06.792043+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:09:06.792043+0000","last_undegraded":"2024-09-15T00:09:06.792043+0000","last_fullsized":"2024-09-15T00:09:06.792043+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"27'12","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.341364+0000","last_change":"2024-09-15T00:09:01.979030+0000","last_active":"2024-09-15T00:09:07.341364+0000","last_peered":"2024-09-15T00:09:07.341364+0000","last_clean":"2024-09-15T00:09:07.341364+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:09:07.341364+0000","last_undegraded":"2024-09-15T00:09:07.341364+0000","last_fullsized":"2024-09-15T00:09:07.341364+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:12:02.235167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059204099999999996,"stat_sum":{"num_bytes":330752,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":10,"num_read_kb":18,"num_write":9,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.789970+0000","last_change":"2024-09-15T00:08:52.835997+0000","last_active":"2024-09-15T00:09:06.789970+0000","last_peered":"2024-09-15T00:09:06.789970+0000","last_clean":"2024-09-15T00:09:06.789970+0000","last_became_active":"2024-09-15T00:08:51.070763+0000","last_became_peered":"2024-09-15T00:08:51.070763+0000","last_unstale":"2024-09-15T00:09:06.789970+0000","last_undegraded":"2024-09-15T00:09:06.789970+0000","last_fullsized":"2024-09-15T00:09:06.789970+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:58.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'82","reported_seq":1161,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.385001+0000","last_change":"2024-09-15T00:09:01.958574+0000","last_active":"2024-09-15T00:09:07.385001+0000","last_peered":"2024-09-15T00:09:07.385001+0000","last_clean":"2024-09-15T00:09:07.385001+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:09:07.385001+0000","last_undegraded":"2024-09-15T00:09:07.385001+0000","last_fullsized":"2024-09-15T00:09:07.385001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:46:01.038754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070311699999999996,"stat_sum":{"num_bytes":1048576,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":2158,"num_read_kb":2334,"num_write":41,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.792278+0000","last_change":"2024-09-15T00:08:52.842966+0000","last_active":"2024-09-15T00:09:06.792278+0000","last_peered":"2024-09-15T00:09:06.792278+0000","last_clean":"2024-09-15T00:09:06.792278+0000","last_became_active":"2024-09-15T00:08:51.070528+0000","last_became_peered":"2024-09-15T00:08:51.070528+0000","last_unstale":"2024-09-15T00:09:06.792278+0000","last_undegraded":"2024-09-15T00:09:06.792278+0000","last_fullsized":"2024-09-15T00:09:06.792278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:52:05.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"27'17","reported_seq":246,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.413320+0000","last_change":"2024-09-15T00:09:01.965506+0000","last_active":"2024-09-15T00:09:07.413320+0000","last_peered":"2024-09-15T00:09:07.413320+0000","last_clean":"2024-09-15T00:09:07.413320+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:09:07.413320+0000","last_undegraded":"2024-09-15T00:09:07.413320+0000","last_fullsized":"2024-09-15T00:09:07.413320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:59.235867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001375294,"stat_sum":{"num_bytes":1384976,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":212,"num_read_kb":1094,"num_write":15,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":189,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.414706+0000","last_change":"2024-09-15T00:09:01.960514+0000","last_active":"2024-09-15T00:09:07.414706+0000","last_peered":"2024-09-15T00:09:07.414706+0000","last_clean":"2024-09-15T00:09:07.414706+0000","last_became_active":"2024-09-15T00:08:59.944404+0000","last_became_peered":"2024-09-15T00:08:59.944404+0000","last_unstale":"2024-09-15T00:09:07.414706+0000","last_undegraded":"2024-09-15T00:09:07.414706+0000","last_fullsized":"2024-09-15T00:09:07.414706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:51.803690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037381900000000001,"stat_sum":{"num_bytes":2449920,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":151,"num_read_kb":1087,"num_write":15,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.808902+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:09:06.808902+0000","last_peered":"2024-09-15T00:09:06.808902+0000","last_clean":"2024-09-15T00:09:06.808902+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:09:06.808902+0000","last_undegraded":"2024-09-15T00:09:06.808902+0000","last_fullsized":"2024-09-15T00:09:06.808902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"27'7","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.439184+0000","last_change":"2024-09-15T00:09:01.958075+0000","last_active":"2024-09-15T00:09:07.439184+0000","last_peered":"2024-09-15T00:09:07.439184+0000","last_clean":"2024-09-15T00:09:07.439184+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:09:07.439184+0000","last_undegraded":"2024-09-15T00:09:07.439184+0000","last_fullsized":"2024-09-15T00:09:07.439184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:29.650790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000567013,"stat_sum":{"num_bytes":1048656,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":102,"num_write":9,"num_write_kb":1067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.789519+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:09:06.789519+0000","last_peered":"2024-09-15T00:09:06.789519+0000","last_clean":"2024-09-15T00:09:06.789519+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:09:06.789519+0000","last_undegraded":"2024-09-15T00:09:06.789519+0000","last_fullsized":"2024-09-15T00:09:06.789519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"27'11","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.439329+0000","last_change":"2024-09-15T00:09:01.969727+0000","last_active":"2024-09-15T00:09:07.439329+0000","last_peered":"2024-09-15T00:09:07.439329+0000","last_clean":"2024-09-15T00:09:07.439329+0000","last_became_active":"2024-09-15T00:08:59.942545+0000","last_became_peered":"2024-09-15T00:08:59.942545+0000","last_unstale":"2024-09-15T00:09:07.439329+0000","last_undegraded":"2024-09-15T00:09:07.439329+0000","last_fullsized":"2024-09-15T00:09:07.439329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:45:34.167112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000355144,"stat_sum":{"num_bytes":1213952,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":30,"num_read_kb":232,"num_write":11,"num_write_kb":891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.789630+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:09:06.789630+0000","last_peered":"2024-09-15T00:09:06.789630+0000","last_clean":"2024-09-15T00:09:06.789630+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:09:06.789630+0000","last_undegraded":"2024-09-15T00:09:06.789630+0000","last_fullsized":"2024-09-15T00:09:06.789630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'17","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.188738+0000","last_change":"2024-09-15T00:09:01.958272+0000","last_active":"2024-09-15T00:09:07.188738+0000","last_peered":"2024-09-15T00:09:07.188738+0000","last_clean":"2024-09-15T00:09:07.188738+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:09:07.188738+0000","last_undegraded":"2024-09-15T00:09:07.188738+0000","last_fullsized":"2024-09-15T00:09:07.188738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:48:24.250392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056893999999999998,"stat_sum":{"num_bytes":81936,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":25,"num_read_kb":13,"num_write":15,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.808922+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:09:06.808922+0000","last_peered":"2024-09-15T00:09:06.808922+0000","last_clean":"2024-09-15T00:09:06.808922+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:09:06.808922+0000","last_undegraded":"2024-09-15T00:09:06.808922+0000","last_fullsized":"2024-09-15T00:09:06.808922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":188,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.789734+0000","last_change":"2024-09-15T00:08:44.804415+0000","last_active":"2024-09-15T00:09:06.789734+0000","last_peered":"2024-09-15T00:09:06.789734+0000","last_clean":"2024-09-15T00:09:06.789734+0000","last_became_active":"2024-09-15T00:08:44.804011+0000","last_became_peered":"2024-09-15T00:08:44.804011+0000","last_unstale":"2024-09-15T00:09:06.789734+0000","last_undegraded":"2024-09-15T00:09:06.789734+0000","last_fullsized":"2024-09-15T00:09:06.789734+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:12.388733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"27'20","reported_seq":201,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.439492+0000","last_change":"2024-09-15T00:09:01.978947+0000","last_active":"2024-09-15T00:09:07.439492+0000","last_peered":"2024-09-15T00:09:07.439492+0000","last_clean":"2024-09-15T00:09:07.439492+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:09:07.439492+0000","last_undegraded":"2024-09-15T00:09:07.439492+0000","last_fullsized":"2024-09-15T00:09:07.439492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:29:23.426311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051427500000000002,"stat_sum":{"num_bytes":851968,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":216,"num_read_kb":632,"num_write":15,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"26'8","reported_seq":336,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.428193+0000","last_change":"2024-09-15T00:09:01.960508+0000","last_active":"2024-09-15T00:09:07.428193+0000","last_peered":"2024-09-15T00:09:07.428193+0000","last_clean":"2024-09-15T00:09:07.428193+0000","last_became_active":"2024-09-15T00:08:59.945869+0000","last_became_peered":"2024-09-15T00:08:59.945869+0000","last_unstale":"2024-09-15T00:09:07.428193+0000","last_undegraded":"2024-09-15T00:09:07.428193+0000","last_fullsized":"2024-09-15T00:09:07.428193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:24:52.190590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031568600000000003,"stat_sum":{"num_bytes":1835008,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":304,"num_read_kb":1664,"num_write":6,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'6","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.011362+0000","last_change":"2024-09-15T00:09:01.953327+0000","last_active":"2024-09-15T00:09:07.011362+0000","last_peered":"2024-09-15T00:09:07.011362+0000","last_clean":"2024-09-15T00:09:07.011362+0000","last_became_active":"2024-09-15T00:08:59.942796+0000","last_became_peered":"2024-09-15T00:08:59.942796+0000","last_unstale":"2024-09-15T00:09:07.011362+0000","last_undegraded":"2024-09-15T00:09:07.011362+0000","last_fullsized":"2024-09-15T00:09:07.011362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:03:53.060052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035268300000000001,"stat_sum":{"num_bytes":147,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":12,"num_read_kb":7,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"26'15","reported_seq":214,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.370999+0000","last_change":"2024-09-15T00:09:01.958373+0000","last_active":"2024-09-15T00:09:07.370999+0000","last_peered":"2024-09-15T00:09:07.370999+0000","last_clean":"2024-09-15T00:09:07.370999+0000","last_became_active":"2024-09-15T00:08:59.941757+0000","last_became_peered":"2024-09-15T00:08:59.941757+0000","last_unstale":"2024-09-15T00:09:07.370999+0000","last_undegraded":"2024-09-15T00:09:07.370999+0000","last_fullsized":"2024-09-15T00:09:07.370999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:33.722246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071565300000000001,"stat_sum":{"num_bytes":2129920,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":175,"num_read_kb":1168,"num_write":14,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"27'96","reported_seq":127,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.885652+0000","last_change":"2024-09-15T00:09:01.964756+0000","last_active":"2024-09-15T00:09:06.885652+0000","last_peered":"2024-09-15T00:09:06.885652+0000","last_clean":"2024-09-15T00:09:06.885652+0000","last_became_active":"2024-09-15T00:08:59.946686+0000","last_became_peered":"2024-09-15T00:08:59.946686+0000","last_unstale":"2024-09-15T00:09:06.885652+0000","last_undegraded":"2024-09-15T00:09:06.885652+0000","last_fullsized":"2024-09-15T00:09:06.885652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:34:50.337149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092954400000000003,"stat_sum":{"num_bytes":180772,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":409,"num_read_kb":319,"num_write":145,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"26'5","reported_seq":98,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.411317+0000","last_change":"2024-09-15T00:09:01.953837+0000","last_active":"2024-09-15T00:09:07.411317+0000","last_peered":"2024-09-15T00:09:07.411317+0000","last_clean":"2024-09-15T00:09:07.411317+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:09:07.411317+0000","last_undegraded":"2024-09-15T00:09:07.411317+0000","last_fullsized":"2024-09-15T00:09:07.411317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:50:56.502827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078421999999999997,"stat_sum":{"num_bytes":1921536,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":66,"num_read_kb":202,"num_write":4,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"27'13","reported_seq":388,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.434723+0000","last_change":"2024-09-15T00:09:01.965099+0000","last_active":"2024-09-15T00:09:07.434723+0000","last_peered":"2024-09-15T00:09:07.434723+0000","last_clean":"2024-09-15T00:09:07.434723+0000","last_became_active":"2024-09-15T00:08:59.945750+0000","last_became_peered":"2024-09-15T00:08:59.945750+0000","last_unstale":"2024-09-15T00:09:07.434723+0000","last_undegraded":"2024-09-15T00:09:07.434723+0000","last_fullsized":"2024-09-15T00:09:07.434723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:49:06.841025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010923459999999999,"stat_sum":{"num_bytes":2944000,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":354,"num_read_kb":2244,"num_write":10,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"27'3","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.397876+0000","last_change":"2024-09-15T00:09:01.965290+0000","last_active":"2024-09-15T00:09:07.397876+0000","last_peered":"2024-09-15T00:09:07.397876+0000","last_clean":"2024-09-15T00:09:07.397876+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:09:07.397876+0000","last_undegraded":"2024-09-15T00:09:07.397876+0000","last_fullsized":"2024-09-15T00:09:07.397876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:59:29.956599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001242525,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":12,"num_write":3,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"27'4","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.378000+0000","last_change":"2024-09-15T00:09:01.964202+0000","last_active":"2024-09-15T00:09:07.378000+0000","last_peered":"2024-09-15T00:09:07.378000+0000","last_clean":"2024-09-15T00:09:07.378000+0000","last_became_active":"2024-09-15T00:08:59.945540+0000","last_became_peered":"2024-09-15T00:08:59.945540+0000","last_unstale":"2024-09-15T00:09:07.378000+0000","last_undegraded":"2024-09-15T00:09:07.378000+0000","last_fullsized":"2024-09-15T00:09:07.378000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:19:13.090172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041955600000000002,"stat_sum":{"num_bytes":364544,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":5,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.311285+0000","last_change":"2024-09-15T00:09:01.953573+0000","last_active":"2024-09-15T00:09:07.311285+0000","last_peered":"2024-09-15T00:09:07.311285+0000","last_clean":"2024-09-15T00:09:07.311285+0000","last_became_active":"2024-09-15T00:08:59.942379+0000","last_became_peered":"2024-09-15T00:08:59.942379+0000","last_unstale":"2024-09-15T00:09:07.311285+0000","last_undegraded":"2024-09-15T00:09:07.311285+0000","last_fullsized":"2024-09-15T00:09:07.311285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:12:39.119693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067808099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":7,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'72","reported_seq":103,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:06.898081+0000","last_change":"2024-09-15T00:09:01.969823+0000","last_active":"2024-09-15T00:09:06.898081+0000","last_peered":"2024-09-15T00:09:06.898081+0000","last_clean":"2024-09-15T00:09:06.898081+0000","last_became_active":"2024-09-15T00:08:59.946308+0000","last_became_peered":"2024-09-15T00:08:59.946308+0000","last_unstale":"2024-09-15T00:09:06.898081+0000","last_undegraded":"2024-09-15T00:09:06.898081+0000","last_fullsized":"2024-09-15T00:09:06.898081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:22.326250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041010299999999999,"stat_sum":{"num_bytes":327737,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":360,"num_read_kb":292,"num_write":126,"num_write_kb":175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"27'21","reported_seq":188,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T00:09:07.420349+0000","last_change":"2024-09-15T00:09:01.953644+0000","last_active":"2024-09-15T00:09:07.420349+0000","last_peered":"2024-09-15T00:09:07.420349+0000","last_clean":"2024-09-15T00:09:07.420349+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:09:07.420349+0000","last_undegraded":"2024-09-15T00:09:07.420349+0000","last_fullsized":"2024-09-15T00:09:07.420349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:18.541304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061215200000000003,"stat_sum":{"num_bytes":1064960,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":145,"num_read_kb":861,"num_write":18,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":41883638,"num_objects":150,"num_object_clones":41,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":39,"num_read":10279,"num_read_kb":30385,"num_write":928,"num_write_kb":9985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2293760,"data_stored":14254060,"data_compressed":429724,"data_compressed_allocated":1261568,"data_compressed_original":13475840,"omap_allocated":0,"internal_metadata":0},"log_size":837,"ondisk_log_size":837,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640264,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":497363,"data_compressed":85718,"data_compressed_allocated":73728,"data_compressed_original":237568,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640264,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":608582,"data_compressed":18224,"data_compressed_allocated":61440,"data_compressed_original":360448,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640265,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":1243895,"data_compressed":111480,"data_compressed_allocated":139264,"data_compressed_original":1007616,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640264,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":630784,"data_stored":3324507,"data_compressed":46859,"data_compressed_allocated":241664,"data_compressed_original":3149824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640264,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":528384,"data_stored":1420965,"data_compressed":13731,"data_compressed_allocated":98304,"data_compressed_original":1220608,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640265,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":929792,"data_stored":4942984,"data_compressed":80643,"data_compressed_allocated":385024,"data_compressed_original":4632576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":4249197,"data_compressed":72115,"data_compressed_allocated":348160,"data_compressed_original":4059136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":1317527,"data_compressed":13731,"data_compressed_allocated":98304,"data_compressed_original":1220608,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":3221088,"data_compressed":46859,"data_compressed_allocated":241664,"data_compressed_original":3149824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":550108,"data_compressed":102952,"data_compressed_allocated":102400,"data_compressed_original":434176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":3498265,"data_compressed":74090,"data_compressed_allocated":299008,"data_compressed_original":3350528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":1417875,"data_compressed":119977,"data_compressed_allocated":172032,"data_compressed_original":1261568,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:09:11.708 INFO:tasks.thrashosds.thrasher:Setting 3.c to [4, 5, 1, 0] 2024-09-15T00:09:11.708 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.c', '4', '5', '1', '0'] 2024-09-15T00:09:11.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.c 4 5 1 0 2024-09-15T00:09:11.742 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:11.740+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:976 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355630+0000", 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4158949759999997, 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014900093999999999, 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 976 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355630+0000", 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355630+0000", 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355631+0000", 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355632+0000", 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.716e-06 2024-09-15T00:09:11.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355639+0000", 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355643+0000", 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2400000000000001e-06 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370458+0000", 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014815439 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370494+0000", 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5988999999999997e-05 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370530+0000", 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6018e-05 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.778 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:978 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 221184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355649+0000", 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4158761750000002, 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014981912999999999, 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 978 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355649+0000", 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355649+0000", 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355649+0000", 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355651+0000", 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.733e-06 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355657+0000", 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1489999999999999e-06 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355662+0000", 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7489999999999998e-06 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370555+0000", 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014893356 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370591+0000", 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5722999999999997e-05 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370631+0000", 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9807e-05 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:980 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355668+0000", 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4158571249999996, 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015042632, 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 980 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355668+0000", 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355668+0000", 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355668+0000", 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0899999999999997e-07 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355670+0000", 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355676+0000", 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8309999999999997e-06 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355681+0000", 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1549999999999998e-06 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370643+0000", 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014962191999999999 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370676+0000", 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3042e-05 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370711+0000", 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4498000000000001e-05 2024-09-15T00:09:11.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:982 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355687+0000", 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4158381980000003, 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015125899, 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 982 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355687+0000", 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355687+0000", 2024-09-15T00:09:11.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355687+0000", 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355689+0000", 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.731e-06 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355695+0000", 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355700+0000", 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.758e-06 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370735+0000", 2024-09-15T00:09:11.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015035389 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370772+0000", 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6480000000000003e-05 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370813+0000", 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1022999999999997e-05 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:984 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355706+0000", 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4158191349999996, 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015314568000000001, 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 984 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355706+0000", 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355706+0000", 2024-09-15T00:09:11.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355706+0000", 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355708+0000", 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7409999999999999e-06 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355714+0000", 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.037e-06 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355719+0000", 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.532e-06 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370858+0000", 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015139379 2024-09-15T00:09:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370943+0000", 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5043000000000006e-05 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371021+0000", 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7438e-05 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:986 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355953+0000", 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4155726729999998, 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015173450999999999, 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 986 2024-09-15T00:09:11.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355953+0000", 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355953+0000", 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355953+0000", 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2300000000000001e-07 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355956+0000", 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0489999999999999e-06 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355963+0000", 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1980000000000004e-06 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355969+0000", 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7989999999999999e-06 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371050+0000", 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015080766000000001 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371085+0000", 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:09:11.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371126+0000", 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1131000000000003e-05 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:988 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 303104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355980+0000", 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4155448809999998, 2024-09-15T00:09:11.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015343035, 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 988 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355980+0000", 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355980+0000", 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355981+0000", 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355983+0000", 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355989+0000", 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1569999999999997e-06 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355994+0000", 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0610000000000003e-06 2024-09-15T00:09:11.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371169+0000", 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015174488999999999 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371247+0000", 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8239999999999996e-05 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371323+0000", 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6588999999999995e-05 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:991 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 319488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4155250969999997, 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015516135, 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 991 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5400000000000002e-07 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356008+0000", 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8679999999999999e-06 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356015+0000", 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2809999999999997e-06 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356019+0000", 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371364+0000", 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015344902000000001 2024-09-15T00:09:11.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371439+0000", 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4666999999999995e-05 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371516+0000", 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7247000000000001e-05 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:993 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356039+0000", 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4154865819999998, 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01567669, 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 993 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356039+0000", 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356039+0000", 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356039+0000", 2024-09-15T00:09:11.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8800000000000003e-07 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356041+0000", 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.17e-06 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356048+0000", 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7070000000000003e-06 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356053+0000", 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8949999999999997e-06 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371567+0000", 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015514457000000001 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371643+0000", 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6148000000000003e-05 2024-09-15T00:09:11.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371715+0000", 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1824999999999998e-05 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:995 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356059+0000", 2024-09-15T00:09:11.816 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4154660049999999, 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015775739, 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 995 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356059+0000", 2024-09-15T00:09:11.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356059+0000", 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356060+0000", 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356061+0000", 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.874e-06 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356067+0000", 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8200000000000002e-06 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356072+0000", 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5070000000000002e-06 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371751+0000", 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015678877000000001 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371793+0000", 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1822e-05 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371835+0000", 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2410999999999998e-05 2024-09-15T00:09:11.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:996 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356077+0000", 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4154482240000004, 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015873707000000001, 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 996 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356077+0000", 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356077+0000", 2024-09-15T00:09:11.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356077+0000", 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.39e-07 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356079+0000", 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7710000000000001e-06 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356085+0000", 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7379999999999999e-06 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356089+0000", 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371850+0000", 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015761126 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371904+0000", 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3390999999999998e-05 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371951+0000", 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6820000000000002e-05 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.825 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:999 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356094+0000", 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4154307270000004, 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015973820999999999, 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 999 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.826 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356094+0000", 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356094+0000", 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356095+0000", 2024-09-15T00:09:11.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.27e-07 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356097+0000", 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356103+0000", 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0739999999999998e-06 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356108+0000", 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5889999999999996e-06 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371977+0000", 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015869007000000001 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372018+0000", 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0525999999999998e-05 2024-09-15T00:09:11.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372068+0000", 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0735999999999997e-05 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1000 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.415412109, 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016143586000000001, 2024-09-15T00:09:11.830 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1000 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:11.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1e-07 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356115+0000", 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6759999999999999e-06 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356120+0000", 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2870000000000004e-06 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356125+0000", 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4750000000000004e-06 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372106+0000", 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015981278000000002 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372187+0000", 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0630999999999994e-05 2024-09-15T00:09:11.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372257+0000", 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9929000000000001e-05 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1003 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356131+0000", 2024-09-15T00:09:11.835 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4153937890000003, 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016310376000000001, 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1003 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356131+0000", 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356131+0000", 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356132+0000", 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356134+0000", 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356140+0000", 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4160000000000001e-06 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356145+0000", 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6059999999999996e-06 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372289+0000", 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016144308999999999 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372335+0000", 2024-09-15T00:09:11.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.562e-05 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372442+0000", 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106783 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1004 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356151+0000", 2024-09-15T00:09:11.840 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4153741310000001, 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016474098999999999, 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1004 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356151+0000", 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356151+0000", 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356151+0000", 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356153+0000", 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9539999999999998e-06 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356159+0000", 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0809999999999999e-06 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356164+0000", 2024-09-15T00:09:11.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4449999999999998e-06 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372464+0000", 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016299783000000002 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372547+0000", 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3015000000000004e-05 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372625+0000", 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8471000000000001e-05 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1006 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4153553710000004, 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016642545000000002, 2024-09-15T00:09:11.845 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1006 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356172+0000", 2024-09-15T00:09:11.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356178+0000", 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0039999999999996e-06 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356183+0000", 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7509999999999999e-06 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372665+0000", 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016482159 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372740+0000", 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4592999999999995e-05 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372812+0000", 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2652999999999997e-05 2024-09-15T00:09:11.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1008 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356188+0000", 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4153370609999998, 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016831110999999999, 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1008 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356188+0000", 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356188+0000", 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356188+0000", 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356190+0000", 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.838e-06 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356196+0000", 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8130000000000001e-06 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356201+0000", 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5170000000000001e-06 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372863+0000", 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016661974999999999 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372943+0000", 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0055000000000005e-05 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373019+0000", 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6607999999999996e-05 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1011 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356206+0000", 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4153196010000002, 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016992022999999998, 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1011 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356206+0000", 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356206+0000", 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356206+0000", 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2000000000000001e-07 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356208+0000", 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.733e-06 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356213+0000", 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6779999999999997e-06 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356218+0000", 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4920000000000004e-06 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373057+0000", 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016839175000000001 2024-09-15T00:09:11.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373128+0000", 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0994000000000005e-05 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373198+0000", 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9630999999999998e-05 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.859 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1014 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356223+0000", 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4153025189999999, 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017112849999999999, 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1014 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356223+0000", 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356223+0000", 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356223+0000", 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356225+0000", 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.759e-06 2024-09-15T00:09:11.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6679999999999998e-06 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356235+0000", 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6489999999999999e-06 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373233+0000", 2024-09-15T00:09:11.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016997486999999999 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373288+0000", 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5714999999999998e-05 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373336+0000", 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7240000000000002e-05 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1015 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356250+0000", 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4152755179999996, 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017247518, 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1015 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356250+0000", 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356250+0000", 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356250+0000", 2024-09-15T00:09:11.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999998e-07 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356252+0000", 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1179999999999999e-06 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356258+0000", 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9580000000000004e-06 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:11.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356263+0000", 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0919999999999997e-06 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373361+0000", 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017098031 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373431+0000", 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9628000000000005e-05 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:11.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373497+0000", 2024-09-15T00:09:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6112000000000007e-05 2024-09-15T00:09:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:11.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:11.869 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:11.871 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:09:11.871 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:11.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:11.868+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:11.874 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone5 2024-09-15T00:09:11.971 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:11.969+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:12.018 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:12.019 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:12.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:12.072 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:12.071+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:12.133 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:12.133 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:12.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:12.185 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:12.184+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:12.201 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-15T00:09:12.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:12.249 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:12.249 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:12.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:12.249 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:12.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.249 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1967 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 851968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.249 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.407321+0000", 2024-09-15T00:09:12.249 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8392314150000004, 2024-09-15T00:09:12.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014035943, 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1967 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407321+0000", 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407321+0000", 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407321+0000", 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407323+0000", 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:09:12.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407329+0000", 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3400000000000003e-06 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407334+0000", 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421203+0000", 2024-09-15T00:09:12.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013868500000000001 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421287+0000", 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4727000000000004e-05 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421357+0000", 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9444999999999997e-05 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.254 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2057 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.254 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409794+0000", 2024-09-15T00:09:12.254 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8367580029999999, 2024-09-15T00:09:12.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014223550999999999, 2024-09-15T00:09:12.254 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.254 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.254 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2057 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409794+0000", 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409794+0000", 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409795+0000", 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409797+0000", 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409802+0000", 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8900000000000004e-06 2024-09-15T00:09:12.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409807+0000", 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5260000000000004e-06 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423810+0000", 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014002979 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423907+0000", 2024-09-15T00:09:12.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7378000000000002e-05 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424018+0000", 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011037200000000001 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2059 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409813+0000", 2024-09-15T00:09:12.258 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8367393119999997, 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01438946, 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2059 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409813+0000", 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409813+0000", 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409813+0000", 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409815+0000", 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9479999999999998e-06 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409821+0000", 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9410000000000004e-06 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409834+0000", 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2692e-05 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424042+0000", 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014208508999999999 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424105+0000", 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2310000000000005e-05 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424202+0000", 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000004e-05 2024-09-15T00:09:12.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2060 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409842+0000", 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8367103, 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014486786, 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2060 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409842+0000", 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409842+0000", 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409842+0000", 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409849+0000", 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.798e-06 2024-09-15T00:09:12.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409856+0000", 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0190000000000001e-06 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409861+0000", 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9799999999999998e-06 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424234+0000", 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014372558000000001 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424276+0000", 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1805999999999999e-05 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424329+0000", 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3103999999999998e-05 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2075 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 663552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409986+0000", 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8365665230000001, 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014495766, 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2075 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409986+0000", 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409986+0000", 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409986+0000", 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409988+0000", 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2699999999999999e-06 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409995+0000", 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2840000000000003e-06 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411222+0000", 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00122773 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424364+0000", 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01314191 2024-09-15T00:09:12.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424428+0000", 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3304000000000002e-05 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424481+0000", 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3832000000000002e-05 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2110 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411468+0000", 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8350844540000004, 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014152062999999999, 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2110 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411468+0000", 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411468+0000", 2024-09-15T00:09:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411468+0000", 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1600000000000001e-07 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411470+0000", 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7239999999999999e-06 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411475+0000", 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3880000000000002e-06 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411478+0000", 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8500000000000004e-06 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425489+0000", 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014010788 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425560+0000", 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0698999999999995e-05 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425620+0000", 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9898000000000001e-05 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.277 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2111 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411483+0000", 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8350694030000003, 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014388593999999999, 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2111 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411483+0000", 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411483+0000", 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411484+0000", 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7899999999999997e-07 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411485+0000", 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.657e-06 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411490+0000", 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3320000000000002e-06 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411493+0000", 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425678+0000", 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014184772999999999 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425764+0000", 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5965999999999997e-05 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425871+0000", 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107187 2024-09-15T00:09:12.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2126 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411589+0000", 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8349634579999998, 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014501538, 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2126 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411589+0000", 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411589+0000", 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411590+0000", 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2200000000000003e-07 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411598+0000", 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411602+0000", 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4889999999999997e-06 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411606+0000", 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-06 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425908+0000", 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014302017 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425989+0000", 2024-09-15T00:09:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0887e-05 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426090+0000", 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101003 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2127 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411611+0000", 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.834941122, 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014674328, 2024-09-15T00:09:12.286 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2127 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411611+0000", 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411611+0000", 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411612+0000", 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9899999999999999e-07 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411613+0000", 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411618+0000", 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.25e-06 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411622+0000", 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.092e-06 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426119+0000", 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014497642 2024-09-15T00:09:12.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426199+0000", 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9221999999999995e-05 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426285+0000", 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.674e-05 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2130 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411626+0000", 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8349259619999998, 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014846062, 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2130 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411626+0000", 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411626+0000", 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411627+0000", 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3099999999999996e-07 2024-09-15T00:09:12.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411629+0000", 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.844e-06 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411633+0000", 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2200000000000003e-06 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411637+0000", 2024-09-15T00:09:12.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8089999999999999e-06 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426320+0000", 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014683018000000001 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426390+0000", 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0424000000000002e-05 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426472+0000", 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1916000000000006e-05 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2146 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411732+0000", 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8348204619999997, 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014971366999999999, 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2146 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411732+0000", 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411732+0000", 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411733+0000", 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.003e-06 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411734+0000", 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.229e-06 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411738+0000", 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5609999999999999e-06 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411746+0000", 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.337e-06 2024-09-15T00:09:12.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426495+0000", 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014749516000000001 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426593+0000", 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589000000000004e-05 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426703+0000", 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110132 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2149 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411752+0000", 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8347998790000002, 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015143592000000001, 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.300 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2149 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411752+0000", 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411752+0000", 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411753+0000", 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411759+0000", 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411766+0000", 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8349999999999998e-06 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411770+0000", 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.408e-06 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426729+0000", 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014959336 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426801+0000", 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1607999999999996e-05 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426896+0000", 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5030000000000003e-05 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2165 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411863+0000", 2024-09-15T00:09:12.304 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8346892849999996, 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015243906999999999, 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2165 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411863+0000", 2024-09-15T00:09:12.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411863+0000", 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411863+0000", 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8699999999999995e-07 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411865+0000", 2024-09-15T00:09:12.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9829999999999999e-06 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411871+0000", 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9730000000000002e-06 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411877+0000", 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1780000000000002e-06 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426929+0000", 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015051981000000001 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427017+0000", 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8602000000000003e-05 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427107+0000", 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9603000000000003e-05 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2168 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411883+0000", 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8346688789999996, 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015420680000000001, 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2168 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411883+0000", 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411883+0000", 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411884+0000", 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2300000000000002e-07 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411886+0000", 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8980000000000001e-06 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411892+0000", 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0020000000000003e-06 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411896+0000", 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427144+0000", 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015247993 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427235+0000", 2024-09-15T00:09:12.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1372000000000005e-05 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427304+0000", 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8533000000000003e-05 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2169 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411902+0000", 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8346497040000003, 2024-09-15T00:09:12.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015568181, 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2169 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411902+0000", 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411902+0000", 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411903+0000", 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4199999999999996e-07 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411905+0000", 2024-09-15T00:09:12.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411911+0000", 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1999999999999999e-06 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411916+0000", 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3299999999999998e-06 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427338+0000", 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015421163 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427404+0000", 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5882000000000004e-05 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427471+0000", 2024-09-15T00:09:12.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7144999999999995e-05 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2184 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 557056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412040+0000", 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8345124620000002, 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015608746999999999, 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2184 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412040+0000", 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412040+0000", 2024-09-15T00:09:12.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412040+0000", 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7500000000000001e-07 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412048+0000", 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8010000000000004e-06 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412054+0000", 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5239999999999997e-06 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412059+0000", 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427510+0000", 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015450445 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427578+0000", 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8533000000000003e-05 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427648+0000", 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0099999999999996e-05 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2185 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.322 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412067+0000", 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8344855610000002, 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015787700000000002, 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2185 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412067+0000", 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412067+0000", 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412067+0000", 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5000000000000003e-07 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412069+0000", 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2350000000000002e-06 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412075+0000", 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.913e-06 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412080+0000", 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7149999999999997e-06 2024-09-15T00:09:12.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427687+0000", 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015606843 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427766+0000", 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8835e-05 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427854+0000", 2024-09-15T00:09:12.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8609000000000005e-05 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2187 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 540672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412086+0000", 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8344660419999999, 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015949517, 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2187 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412086+0000", 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412086+0000", 2024-09-15T00:09:12.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412087+0000", 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5900000000000002e-07 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412089+0000", 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9360000000000002e-06 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412094+0000", 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8839999999999999e-06 2024-09-15T00:09:12.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412109+0000", 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4600000000000001e-05 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427872+0000", 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015762524 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427937+0000", 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5191999999999995e-05 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.428036+0000", 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8921999999999999e-05 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2188 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412116+0000", 2024-09-15T00:09:12.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8344359739999998, 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016139355000000001, 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2188 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412116+0000", 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412116+0000", 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412117+0000", 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5300000000000004e-07 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412119+0000", 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.159e-06 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412125+0000", 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1530000000000002e-06 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412130+0000", 2024-09-15T00:09:12.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7620000000000003e-06 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.428061+0000", 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015930693999999999 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.428137+0000", 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6377000000000004e-05 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.428256+0000", 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118657 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2202 3.1a 3:58ff36af:::rbd_data.1129feb48f4b.0000000000000005:4 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8342140840000001, 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014126756000000001, 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:12.336 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2202 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412339+0000", 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412340+0000", 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-15T00:09:12.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412346+0000", 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9900000000000002e-06 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412356+0000", 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6870000000000006e-06 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426297+0000", 2024-09-15T00:09:12.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013940666000000001 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426373+0000", 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6891999999999995e-05 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426465+0000", 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1399999999999999e-05 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:12.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:12.341 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:12.342 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:12.341+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:12.343 INFO:teuthology.orchestra.run.smithi092.stdout:704 trunc from 0x8cbe400 to 0x1ffbc00 2024-09-15T00:09:12.343 INFO:teuthology.orchestra.run.smithi092.stdout:705 write 0x650fa00 thru 0x651dab8 (0xe0b9 bytes) 2024-09-15T00:09:12.343 INFO:teuthology.orchestra.run.smithi092.stdout:708 read 0x1719e00 thru 0x17212be (0x74bf bytes) 2024-09-15T00:09:12.343 INFO:teuthology.orchestra.run.smithi092.stdout:711 write 0x5afa200 thru 0x5afa6dd (0x4de bytes) 2024-09-15T00:09:12.343 INFO:teuthology.orchestra.run.smithi092.stdout:712 read 0x2c2a800 thru 0x2c313ef (0x6bf0 bytes) 2024-09-15T00:09:12.343 INFO:teuthology.orchestra.run.smithi092.stdout:715 trunc from 0x651dc00 to 0xadb3800 2024-09-15T00:09:12.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:12.346 INFO:teuthology.orchestra.run.smithi092.stdout:716 read 0x1ff3000 thru 0x1ffae0f (0x7e10 bytes) 2024-09-15T00:09:12.346 INFO:teuthology.orchestra.run.smithi092.stdout:719 read 0x1e2b800 thru 0x1e3a30f (0xeb10 bytes) 2024-09-15T00:09:12.347 INFO:teuthology.orchestra.run.smithi092.stdout:721 read 0x2fb3c00 thru 0x2fb9aaa (0x5eab bytes) 2024-09-15T00:09:12.348 INFO:teuthology.orchestra.run.smithi092.stdout:723 read 0x51aa200 thru 0x51b08d6 (0x66d7 bytes) 2024-09-15T00:09:12.355 INFO:teuthology.orchestra.run.smithi092.stdout:724 trunc from 0xadb3800 to 0x5388600 2024-09-15T00:09:12.367 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:12.368 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:12.377 INFO:teuthology.orchestra.run.smithi092.stdout:726 punch from 0x2b5a800 to 0x2b68600, (0xde00 bytes) 2024-09-15T00:09:12.377 INFO:teuthology.orchestra.run.smithi092.stdout:727 read 0x4f36c00 thru 0x4f3c015 (0x5416 bytes) 2024-09-15T00:09:12.377 INFO:teuthology.orchestra.run.smithi092.stdout:728 trunc from 0x5388600 to 0x4951800 2024-09-15T00:09:12.386 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:12.386 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:12.394 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xed5a800 2024-09-15T00:09:12.403 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:12.403 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:12.406 INFO:teuthology.orchestra.run.smithi092.stdout:734 trunc from 0x4951800 to 0xed5a800 2024-09-15T00:09:12.415 INFO:teuthology.orchestra.run.smithi092.stdout:735 write 0x2f2ec00 thru 0x2f31cca (0x30cb bytes) 2024-09-15T00:09:12.422 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:12.423 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:12.424 INFO:teuthology.orchestra.run.smithi092.stdout:737 read 0x8a8d600 thru 0x8a970ad (0x9aae bytes) 2024-09-15T00:09:12.424 INFO:teuthology.orchestra.run.smithi092.stdout:739 read 0x8b18a00 thru 0x8b25160 (0xc761 bytes) 2024-09-15T00:09:12.424 INFO:teuthology.orchestra.run.smithi092.stdout:742 punch from 0x49bd400 to 0x49ccc00, (0xf800 bytes) 2024-09-15T00:09:12.424 INFO:teuthology.orchestra.run.smithi092.stdout:745 read 0xc577000 thru 0xc5812c2 (0xa2c3 bytes) 2024-09-15T00:09:12.424 INFO:teuthology.orchestra.run.smithi092.stdout:746 punch from 0xc035400 to 0xc036200, (0xe00 bytes) 2024-09-15T00:09:12.424 INFO:teuthology.orchestra.run.smithi092.stdout:748 write 0xc6c9600 thru 0xc6ca0e5 (0xae6 bytes) 2024-09-15T00:09:12.424 INFO:teuthology.orchestra.run.smithi092.stdout:750 write 0x3edb200 thru 0x3edc177 (0xf78 bytes) 2024-09-15T00:09:12.424 INFO:teuthology.orchestra.run.smithi092.stdout:751 read 0x1a91c00 thru 0x1a9a9c9 (0x8dca bytes) 2024-09-15T00:09:12.425 INFO:teuthology.orchestra.run.smithi092.stdout:752 punch from 0xea14400 to 0xea1a800, (0x6400 bytes) 2024-09-15T00:09:12.434 INFO:teuthology.orchestra.run.smithi092.stdout:753 trunc from 0xed5a800 to 0x6794800 2024-09-15T00:09:12.441 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:12.442 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:12.441+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:12.450 INFO:teuthology.orchestra.run.smithi092.stdout:754 read 0x335a600 thru 0x335e333 (0x3d34 bytes) 2024-09-15T00:09:12.457 INFO:teuthology.orchestra.run.smithi092.stdout:756 read 0x5d8e000 thru 0x5d99128 (0xb129 bytes) 2024-09-15T00:09:12.457 INFO:teuthology.orchestra.run.smithi092.stdout:757 read 0x39dd200 thru 0x39ebae3 (0xe8e4 bytes) 2024-09-15T00:09:12.458 INFO:teuthology.orchestra.run.smithi092.stdout:758 write 0x49af800 thru 0x49b9101 (0x9902 bytes) 2024-09-15T00:09:12.458 INFO:teuthology.orchestra.run.smithi092.stdout:761 read 0x26f1800 thru 0x2700147 (0xe948 bytes) 2024-09-15T00:09:12.460 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:12.460 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:12.474 INFO:teuthology.orchestra.run.smithi092.stdout:762 write 0xb91a000 thru 0xb9259ec (0xb9ed bytes) 2024-09-15T00:09:12.482 INFO:teuthology.orchestra.run.smithi092.stdout:764 punch from 0x8c24c00 to 0x8c2a200, (0x5600 bytes) 2024-09-15T00:09:12.482 INFO:teuthology.orchestra.run.smithi092.stdout:765 punch from 0x9397a00 to 0x939c400, (0x4a00 bytes) 2024-09-15T00:09:12.483 INFO:teuthology.orchestra.run.smithi092.stdout:773 read 0x947fe00 thru 0x948d1cc (0xd3cd bytes) 2024-09-15T00:09:12.483 INFO:teuthology.orchestra.run.smithi092.stdout:776 write 0x4e1e00 thru 0x4e48d6 (0x2ad7 bytes) 2024-09-15T00:09:12.483 INFO:teuthology.orchestra.run.smithi092.stdout:783 punch from 0x2588400 to 0x2595a00, (0xd600 bytes) 2024-09-15T00:09:12.484 INFO:teuthology.orchestra.run.smithi092.stdout:787 read 0x330200 thru 0x334846 (0x4647 bytes) 2024-09-15T00:09:12.484 INFO:teuthology.orchestra.run.smithi092.stdout:788 clone 7 order 22 su 65536 sc 7 2024-09-15T00:09:12.493 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:12.493 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:12.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:12.543 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:12.542+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:12.643 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:12.643 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:12.644 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:12.642+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:12.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:12.745 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:12.744+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:12.755 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:12.755 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:12.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:12.817 INFO:teuthology.orchestra.run.smithi027.stderr:set 3.c pg_upmap_items mapping to [4->5,1->0] 2024-09-15T00:09:12.845 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:12.844+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:12.880 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone6 from 0xb925a00 (overlap 0x1ffbc00) to 0x1663600 2024-09-15T00:09:12.882 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:12.882 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:12.893 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:12.947 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:12.945+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:13.003 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:13.003 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:13.012 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:13.048 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:13.046+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:13.123 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:13.123 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:13.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:13.175 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-15T00:09:13.176 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:13.175+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:13.243 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:13.243 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:13.249 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:13.248+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:13.252 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:13.351 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:13.350+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:13.406 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:13.406 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:13.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:13.457 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:13.456+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:13.553 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:13.552+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:13.562 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:13.563 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:13.571 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:13.587 INFO:teuthology.orchestra.run.smithi092.stdout:789 trunc from 0xb925a00 to 0xba6d400 2024-09-15T00:09:13.612 INFO:teuthology.orchestra.run.smithi092.stdout:791 write 0xdab0800 thru 0xdabf70b (0xef0c bytes) 2024-09-15T00:09:13.613 INFO:teuthology.orchestra.run.smithi092.stdout:792 write 0x2a14800 thru 0x2a22f71 (0xe772 bytes) 2024-09-15T00:09:13.613 INFO:teuthology.orchestra.run.smithi092.stdout:796 read 0x5644200 thru 0x564690a (0x270b bytes) 2024-09-15T00:09:13.614 INFO:teuthology.orchestra.run.smithi092.stdout:801 write 0x3111e00 thru 0x3113a19 (0x1c1a bytes) 2024-09-15T00:09:13.618 INFO:teuthology.orchestra.run.smithi092.stdout:802 trunc from 0xdabf800 to 0xa220000 2024-09-15T00:09:13.653 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:13.652+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:13.661 INFO:teuthology.orchestra.run.smithi092.stdout:803 write 0x336fc00 thru 0x337b566 (0xb967 bytes) 2024-09-15T00:09:13.661 INFO:teuthology.orchestra.run.smithi092.stdout:804 write 0x12a9600 thru 0x12b174d (0x814e bytes) 2024-09-15T00:09:13.662 INFO:teuthology.orchestra.run.smithi092.stdout:806 punch from 0x2db9e00 to 0x2dc0c00, (0x6e00 bytes) 2024-09-15T00:09:13.670 INFO:teuthology.orchestra.run.smithi092.stdout:807 trunc from 0xa220000 to 0x5397600 2024-09-15T00:09:13.716 INFO:teuthology.orchestra.run.smithi092.stdout:808 read 0x1aad000 thru 0x1abc9c6 (0xf9c7 bytes) 2024-09-15T00:09:13.724 INFO:teuthology.orchestra.run.smithi092.stdout:809 read 0x1ac8000 thru 0x1ad632b (0xe32c bytes) 2024-09-15T00:09:13.725 INFO:teuthology.orchestra.run.smithi092.stdout:811 read 0x1d8b400 thru 0x1d8e805 (0x3406 bytes) 2024-09-15T00:09:13.725 INFO:teuthology.orchestra.run.smithi092.stdout:813 read 0xbc5200 thru 0xbc7f3c (0x2d3d bytes) 2024-09-15T00:09:13.726 INFO:teuthology.orchestra.run.smithi092.stdout:816 punch from 0x1fe7200 to 0x1febe00, (0x4c00 bytes) 2024-09-15T00:09:13.726 INFO:teuthology.orchestra.run.smithi092.stdout:817 read 0x4887a00 thru 0x4897856 (0xfe57 bytes) 2024-09-15T00:09:13.736 INFO:teuthology.orchestra.run.smithi092.stdout:818 trunc from 0x5397600 to 0xd593000 2024-09-15T00:09:13.738 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:13.738 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:13.746 INFO:teuthology.orchestra.run.smithi092.stdout:821 read 0xc045e00 thru 0xc052505 (0xc706 bytes) 2024-09-15T00:09:13.749 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:13.753 INFO:teuthology.orchestra.run.smithi092.stdout:822 write 0xa3f1000 thru 0xa3f838b (0x738c bytes) 2024-09-15T00:09:13.753 INFO:teuthology.orchestra.run.smithi092.stdout:823 read 0x31400 thru 0x36e55 (0x5a56 bytes) 2024-09-15T00:09:13.755 INFO:teuthology.orchestra.run.smithi092.stdout:826 read 0x8821e00 thru 0x882b107 (0x9308 bytes) 2024-09-15T00:09:13.755 INFO:teuthology.orchestra.run.smithi092.stdout:829 write 0xac5f400 thru 0xac5fdd6 (0x9d7 bytes) 2024-09-15T00:09:13.755 INFO:teuthology.orchestra.run.smithi092.stdout:830 clone 8 order 23 su 65536 sc 6 2024-09-15T00:09:13.793 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:13.792+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:13.855 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:13.854+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:13.863 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:13.863 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:13.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:09:13.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:13.900 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone7 2024-09-15T00:09:13.957 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:13.956+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:14.029 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:14.029 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:14.040 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:14.081 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:14.080+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:14.144 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:14.144 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:14.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:14.157 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:14.156+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:14.258 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:14.257+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:14.259 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:14.259 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:14.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:14.326 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-15T00:09:14.359 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:14.357+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:14.372 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:14.372 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:14.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:14.460 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:14.459+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:14.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:14.524 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:14.524 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:14.533 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:14.561 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:14.560+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:14.599 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:14.599 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:14.617 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:14.618 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:14.634 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:14.634 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:14.641 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:14.641 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:14.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:14.651 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:14.651 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:14.662 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:14.661+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:14.667 INFO:teuthology.orchestra.run.smithi092.stdout:831 punch from 0x2c87c00 to 0x2c90e00, (0x9200 bytes) 2024-09-15T00:09:14.668 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:14.669 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:14.690 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:14.690 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:14.737 INFO:teuthology.orchestra.run.smithi092.stdout:832 punch from 0x7861a00 to 0x786fe00, (0xe400 bytes) 2024-09-15T00:09:14.753 INFO:teuthology.orchestra.run.smithi092.stdout:834 trunc from 0xd593000 to 0x3a7fa00 2024-09-15T00:09:14.756 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1967 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 851968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.407321+0000", 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3472557749999998, 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014035943, 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1967 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407321+0000", 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407321+0000", 2024-09-15T00:09:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407321+0000", 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407323+0000", 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407329+0000", 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3400000000000003e-06 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.407334+0000", 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6020000000000002e-06 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421203+0000", 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013868500000000001 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421287+0000", 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4727000000000004e-05 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421357+0000", 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9444999999999997e-05 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2057 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409794+0000", 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3447823630000002, 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014223550999999999, 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2057 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409794+0000", 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409794+0000", 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409795+0000", 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409797+0000", 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409802+0000", 2024-09-15T00:09:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8900000000000004e-06 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409807+0000", 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5260000000000004e-06 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423810+0000", 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014002979 2024-09-15T00:09:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423907+0000", 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7378000000000002e-05 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424018+0000", 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011037200000000001 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2059 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409813+0000", 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.344763672, 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01438946, 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2059 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409813+0000", 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409813+0000", 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409813+0000", 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409815+0000", 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9479999999999998e-06 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409821+0000", 2024-09-15T00:09:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9410000000000004e-06 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409834+0000", 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2692e-05 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424042+0000", 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014208508999999999 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424105+0000", 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2310000000000005e-05 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424202+0000", 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000004e-05 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2060 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409842+0000", 2024-09-15T00:09:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3447346600000003, 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014486786, 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2060 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409842+0000", 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409842+0000", 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409842+0000", 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2099999999999997e-07 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409849+0000", 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.798e-06 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409856+0000", 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0190000000000001e-06 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409861+0000", 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9799999999999998e-06 2024-09-15T00:09:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424234+0000", 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014372558000000001 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424276+0000", 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1805999999999999e-05 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424329+0000", 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3103999999999998e-05 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2075 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 663552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409986+0000", 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3445908830000004, 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014495766, 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2075 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409986+0000", 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409986+0000", 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409986+0000", 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409988+0000", 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2699999999999999e-06 2024-09-15T00:09:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409995+0000", 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2840000000000003e-06 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411222+0000", 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00122773 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424364+0000", 2024-09-15T00:09:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01314191 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424428+0000", 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3304000000000002e-05 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424481+0000", 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3832000000000002e-05 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2110 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411468+0000", 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3431088139999998, 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014152062999999999, 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2110 2024-09-15T00:09:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411468+0000", 2024-09-15T00:09:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411468+0000", 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411468+0000", 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1600000000000001e-07 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411470+0000", 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7239999999999999e-06 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411475+0000", 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3880000000000002e-06 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411478+0000", 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8500000000000004e-06 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425489+0000", 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014010788 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425560+0000", 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0698999999999995e-05 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425620+0000", 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9898000000000001e-05 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2111 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411483+0000", 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3430937629999997, 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014388593999999999, 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2111 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411483+0000", 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411483+0000", 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411484+0000", 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7899999999999997e-07 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411485+0000", 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.657e-06 2024-09-15T00:09:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411490+0000", 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3320000000000002e-06 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411493+0000", 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-15T00:09:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425678+0000", 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014184772999999999 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425764+0000", 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5965999999999997e-05 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425871+0000", 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107187 2024-09-15T00:09:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2126 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411589+0000", 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3429878180000001, 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014501538, 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2126 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411589+0000", 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411589+0000", 2024-09-15T00:09:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411590+0000", 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2200000000000003e-07 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411598+0000", 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4300000000000006e-06 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411602+0000", 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4889999999999997e-06 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411606+0000", 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-06 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425908+0000", 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014302017 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.425989+0000", 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0887e-05 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426090+0000", 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101003 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2127 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411611+0000", 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3429654820000003, 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014674328, 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2127 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411611+0000", 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411611+0000", 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411612+0000", 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9899999999999999e-07 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411613+0000", 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5829999999999999e-06 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411618+0000", 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.25e-06 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411622+0000", 2024-09-15T00:09:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.092e-06 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426119+0000", 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014497642 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426199+0000", 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9221999999999995e-05 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426285+0000", 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.674e-05 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2130 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411626+0000", 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3429503220000001, 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014846062, 2024-09-15T00:09:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2130 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411626+0000", 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411626+0000", 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411627+0000", 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3099999999999996e-07 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411629+0000", 2024-09-15T00:09:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.844e-06 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411633+0000", 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2200000000000003e-06 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411637+0000", 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8089999999999999e-06 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426320+0000", 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014683018000000001 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426390+0000", 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0424000000000002e-05 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426472+0000", 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1916000000000006e-05 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2146 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411732+0000", 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.342844822, 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014971366999999999, 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2146 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411732+0000", 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411732+0000", 2024-09-15T00:09:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411733+0000", 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.003e-06 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411734+0000", 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.229e-06 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411738+0000", 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5609999999999999e-06 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411746+0000", 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.337e-06 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426495+0000", 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014749516000000001 2024-09-15T00:09:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426593+0000", 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589000000000004e-05 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426703+0000", 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110132 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2149 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411752+0000", 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3428242389999996, 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015143592000000001, 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2149 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411752+0000", 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411752+0000", 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411753+0000", 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411759+0000", 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411766+0000", 2024-09-15T00:09:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8349999999999998e-06 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411770+0000", 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.408e-06 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426729+0000", 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014959336 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426801+0000", 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1607999999999996e-05 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426896+0000", 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5030000000000003e-05 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2165 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411863+0000", 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3427136449999999, 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015243906999999999, 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2165 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411863+0000", 2024-09-15T00:09:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411863+0000", 2024-09-15T00:09:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411863+0000", 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8699999999999995e-07 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411865+0000", 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9829999999999999e-06 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411871+0000", 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9730000000000002e-06 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411877+0000", 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1780000000000002e-06 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426929+0000", 2024-09-15T00:09:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015051981000000001 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427017+0000", 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8602000000000003e-05 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427107+0000", 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9603000000000003e-05 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2168 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411883+0000", 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3426932389999999, 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015420680000000001, 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2168 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411883+0000", 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411883+0000", 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411884+0000", 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2300000000000002e-07 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411886+0000", 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8980000000000001e-06 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411892+0000", 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0020000000000003e-06 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411896+0000", 2024-09-15T00:09:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-15T00:09:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427144+0000", 2024-09-15T00:09:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015247993 2024-09-15T00:09:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427235+0000", 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1372000000000005e-05 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427304+0000", 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8533000000000003e-05 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2169 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.411902+0000", 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3426740639999997, 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015568181, 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2169 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411902+0000", 2024-09-15T00:09:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411902+0000", 2024-09-15T00:09:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411903+0000", 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4199999999999996e-07 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411905+0000", 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9190000000000002e-06 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411911+0000", 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1999999999999999e-06 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.411916+0000", 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3299999999999998e-06 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427338+0000", 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015421163 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427404+0000", 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5882000000000004e-05 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427471+0000", 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7144999999999995e-05 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2184 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 557056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412040+0000", 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3425368219999996, 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015608746999999999, 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2184 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412040+0000", 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412040+0000", 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412040+0000", 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7500000000000001e-07 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412048+0000", 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8010000000000004e-06 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412054+0000", 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5239999999999997e-06 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412059+0000", 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427510+0000", 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015450445 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427578+0000", 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8533000000000003e-05 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427648+0000", 2024-09-15T00:09:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0099999999999996e-05 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2185 3.9 3:944f13cd:::rbd_data.1129feb48f4b.0000000000000006:4 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412067+0000", 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3425099209999996, 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015787700000000002, 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2185 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412067+0000", 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412067+0000", 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412067+0000", 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5000000000000003e-07 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412069+0000", 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2350000000000002e-06 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412075+0000", 2024-09-15T00:09:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.913e-06 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412080+0000", 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7149999999999997e-06 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427687+0000", 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015606843 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427766+0000", 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8835e-05 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427854+0000", 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8609000000000005e-05 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2187 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 540672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412086+0000", 2024-09-15T00:09:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3424904020000001, 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015949517, 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2187 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412086+0000", 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412086+0000", 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412087+0000", 2024-09-15T00:09:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5900000000000002e-07 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412089+0000", 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9360000000000002e-06 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412094+0000", 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8839999999999999e-06 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412109+0000", 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4600000000000001e-05 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427872+0000", 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015762524 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.427937+0000", 2024-09-15T00:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5191999999999995e-05 2024-09-15T00:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.428036+0000", 2024-09-15T00:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8921999999999999e-05 2024-09-15T00:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2188 3.9 3:90d14c4f:::rbd_data.1129feb48f4b.0000000000000008:4 [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412116+0000", 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3424603340000001, 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016139355000000001, 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2188 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412116+0000", 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412116+0000", 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412117+0000", 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5300000000000004e-07 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412119+0000", 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.159e-06 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412125+0000", 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1530000000000002e-06 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412130+0000", 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7620000000000003e-06 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.428061+0000", 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015930693999999999 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.428137+0000", 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6377000000000004e-05 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.428256+0000", 2024-09-15T00:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118657 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2202 3.1a 3:58ff36af:::rbd_data.1129feb48f4b.0000000000000005:4 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3422384440000004, 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014126756000000001, 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2202 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412339+0000", 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412340+0000", 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.654e-06 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412346+0000", 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9900000000000002e-06 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412356+0000", 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6870000000000006e-06 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426297+0000", 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013940666000000001 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426373+0000", 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6891999999999995e-05 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.426465+0000", 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1399999999999999e-05 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:09:14.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:14.857 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:14.854+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:835 read 0x7b8200 thru 0x7c53f6 (0xd1f7 bytes) 2024-09-15T00:09:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:836 write 0xe4b3400 thru 0xe4c2e37 (0xfa38 bytes) 2024-09-15T00:09:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:838 read 0x948d200 thru 0x948fefd (0x2cfe bytes) 2024-09-15T00:09:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:839 read 0x4747400 thru 0x4750862 (0x9463 bytes) 2024-09-15T00:09:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:840 write 0xb653200 thru 0xb661d67 (0xeb68 bytes) 2024-09-15T00:09:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:847 write 0x1874000 thru 0x187d858 (0x9859 bytes) 2024-09-15T00:09:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:848 write 0x759cc00 thru 0x75a1e41 (0x5242 bytes) 2024-09-15T00:09:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:849 trunc from 0xe4c3000 to 0xdc97000 2024-09-15T00:09:14.880 INFO:teuthology.orchestra.run.smithi092.stdout:850 write 0x5b87c00 thru 0x5b893cc (0x17cd bytes) 2024-09-15T00:09:14.881 INFO:teuthology.orchestra.run.smithi092.stdout:851 trunc from 0xdc97000 to 0xde9ae00 2024-09-15T00:09:14.895 INFO:teuthology.orchestra.run.smithi092.stdout:853 write 0x8f2800 thru 0x8f68ae (0x40af bytes) 2024-09-15T00:09:14.896 INFO:teuthology.orchestra.run.smithi092.stdout:855 write 0x4bb7400 thru 0x4bc6c3f (0xf840 bytes) 2024-09-15T00:09:14.897 INFO:teuthology.orchestra.run.smithi092.stdout:857 write 0x8fbd000 thru 0x8fbf4fb (0x24fc bytes) 2024-09-15T00:09:14.897 INFO:teuthology.orchestra.run.smithi092.stdout:858 read 0x2d49c00 thru 0x2d552b2 (0xb6b3 bytes) 2024-09-15T00:09:14.897 INFO:teuthology.orchestra.run.smithi092.stdout:862 read 0x6c30a00 thru 0x6c407d1 (0xfdd2 bytes) 2024-09-15T00:09:14.899 INFO:teuthology.orchestra.run.smithi092.stdout:863 punch from 0xa51f600 to 0xa52ce00, (0xd800 bytes) 2024-09-15T00:09:14.900 INFO:teuthology.orchestra.run.smithi092.stdout:865 punch from 0xa7c3a00 to 0xa7cf600, (0xbc00 bytes) 2024-09-15T00:09:14.900 INFO:teuthology.orchestra.run.smithi092.stdout:866 punch from 0x80ec600 to 0x80f4800, (0x8200 bytes) 2024-09-15T00:09:14.908 INFO:teuthology.orchestra.run.smithi092.stdout:868 trunc from 0xde9ae00 to 0x6db0a00 2024-09-15T00:09:14.922 INFO:teuthology.orchestra.run.smithi092.stdout:869 flatten 2024-09-15T00:09:14.956 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:14.955+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:14.966 INFO:teuthology.orchestra.run.smithi092.stdout:870 write 0x5b97600 thru 0x5ba4a0f (0xd410 bytes) 2024-09-15T00:09:14.967 INFO:teuthology.orchestra.run.smithi092.stdout:872 punch from 0x2be8e00 to 0x2bf8c00, (0xfe00 bytes) 2024-09-15T00:09:14.967 INFO:teuthology.orchestra.run.smithi092.stdout:876 punch from 0x1d8cc00 to 0x1d8f400, (0x2800 bytes) 2024-09-15T00:09:14.967 INFO:teuthology.orchestra.run.smithi092.stdout:877 write 0x38f5200 thru 0x38fb9ec (0x67ed bytes) 2024-09-15T00:09:14.968 INFO:teuthology.orchestra.run.smithi092.stdout:881 punch from 0x4454e00 to 0x445ae00, (0x6000 bytes) 2024-09-15T00:09:14.969 INFO:teuthology.orchestra.run.smithi092.stdout:887 read 0x36c6a00 thru 0x36c7dac (0x13ad bytes) 2024-09-15T00:09:14.975 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:14.975 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:14.976 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:14.976 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:14.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:14.985 INFO:teuthology.orchestra.run.smithi092.stdout:893 write 0xc284000 thru 0xc286fe6 (0x2fe7 bytes) 2024-09-15T00:09:14.986 INFO:teuthology.orchestra.run.smithi092.stdout:894 write 0x64aa400 thru 0x64ac12a (0x1d2b bytes) 2024-09-15T00:09:14.986 INFO:teuthology.orchestra.run.smithi092.stdout:895 write 0x6426200 thru 0x64308e4 (0xa6e5 bytes) 2024-09-15T00:09:15.000 INFO:teuthology.orchestra.run.smithi092.stdout:896 write 0xdd14400 thru 0xdd20a30 (0xc631 bytes) 2024-09-15T00:09:15.001 INFO:teuthology.orchestra.run.smithi092.stdout:897 write 0x272a800 thru 0x2737f21 (0xd722 bytes) 2024-09-15T00:09:15.014 INFO:teuthology.orchestra.run.smithi092.stdout:899 write 0xe369200 thru 0xe37489e (0xb69f bytes) 2024-09-15T00:09:15.015 INFO:teuthology.orchestra.run.smithi092.stdout:900 read 0x273e400 thru 0x273fca1 (0x18a2 bytes) 2024-09-15T00:09:15.015 INFO:teuthology.orchestra.run.smithi092.stdout:901 punch from 0x492f400 to 0x4932200, (0x2e00 bytes) 2024-09-15T00:09:15.015 INFO:teuthology.orchestra.run.smithi092.stdout:902 read 0x1b35c00 thru 0x1b3b99d (0x5d9e bytes) 2024-09-15T00:09:15.016 INFO:teuthology.orchestra.run.smithi092.stdout:903 write 0xd1e8800 thru 0xd1ee78d (0x5f8e bytes) 2024-09-15T00:09:15.017 INFO:teuthology.orchestra.run.smithi092.stdout:904 punch from 0xd209200 to 0xd20fc00, (0x6a00 bytes) 2024-09-15T00:09:15.017 INFO:teuthology.orchestra.run.smithi092.stdout:908 punch from 0x6f41200 to 0x6f43800, (0x2600 bytes) 2024-09-15T00:09:15.017 INFO:teuthology.orchestra.run.smithi092.stdout:909 write 0x9031c00 thru 0x90417c4 (0xfbc5 bytes) 2024-09-15T00:09:15.017 INFO:teuthology.orchestra.run.smithi092.stdout:910 read 0xdef4800 thru 0xdf0447f (0xfc80 bytes) 2024-09-15T00:09:15.018 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xedcc800 2024-09-15T00:09:15.019 INFO:teuthology.orchestra.run.smithi092.stdout:911 trunc from 0xe374a00 to 0xedcc800 2024-09-15T00:09:15.032 INFO:teuthology.orchestra.run.smithi092.stdout:912 read 0x3df2800 thru 0x3e017f5 (0xeff6 bytes) 2024-09-15T00:09:15.032 INFO:teuthology.orchestra.run.smithi092.stdout:914 write 0x1d37200 thru 0x1d3d37b (0x617c bytes) 2024-09-15T00:09:15.033 INFO:teuthology.orchestra.run.smithi092.stdout:916 read 0x20a2800 thru 0x20a7e51 (0x5652 bytes) 2024-09-15T00:09:15.034 INFO:teuthology.orchestra.run.smithi092.stdout:918 write 0x845e200 thru 0x846b65a (0xd45b bytes) 2024-09-15T00:09:15.034 INFO:teuthology.orchestra.run.smithi092.stdout:919 punch from 0x4730800 to 0x473e800, (0xe000 bytes) 2024-09-15T00:09:15.035 INFO:teuthology.orchestra.run.smithi092.stdout:924 write 0x25c7a00 thru 0x25cbc69 (0x426a bytes) 2024-09-15T00:09:15.036 INFO:teuthology.orchestra.run.smithi092.stdout:926 punch from 0xda7d800 to 0xda8b400, (0xdc00 bytes) 2024-09-15T00:09:15.036 INFO:teuthology.orchestra.run.smithi092.stdout:930 write 0xd449000 thru 0xd44f1ad (0x61ae bytes) 2024-09-15T00:09:15.037 INFO:teuthology.orchestra.run.smithi092.stdout:932 punch from 0x90b2000 to 0x90b7200, (0x5200 bytes) 2024-09-15T00:09:15.037 INFO:teuthology.orchestra.run.smithi092.stdout:933 read 0x5d08800 thru 0x5d15177 (0xc978 bytes) 2024-09-15T00:09:15.037 INFO:teuthology.orchestra.run.smithi092.stdout:936 write 0xa750400 thru 0xa7531df (0x2de0 bytes) 2024-09-15T00:09:15.038 INFO:teuthology.orchestra.run.smithi092.stdout:938 read 0xd4a4a00 thru 0xd4b00d5 (0xb6d6 bytes) 2024-09-15T00:09:15.038 INFO:teuthology.orchestra.run.smithi092.stdout:939 read 0x3841800 thru 0x3848ae0 (0x72e1 bytes) 2024-09-15T00:09:15.038 INFO:teuthology.orchestra.run.smithi092.stdout:944 read 0x99aaa00 thru 0x99b09b1 (0x5fb2 bytes) 2024-09-15T00:09:15.040 INFO:teuthology.orchestra.run.smithi092.stdout:949 write 0xe5b2200 thru 0xe5b857d (0x637e bytes) 2024-09-15T00:09:15.041 INFO:teuthology.orchestra.run.smithi092.stdout:950 write 0x284a00 thru 0x29325a (0xe85b bytes) 2024-09-15T00:09:15.041 INFO:teuthology.orchestra.run.smithi092.stdout:952 write 0x8275400 thru 0x827f9ad (0xa5ae bytes) 2024-09-15T00:09:15.042 INFO:teuthology.orchestra.run.smithi092.stdout:955 read 0x5684a00 thru 0x5693616 (0xec17 bytes) 2024-09-15T00:09:15.042 INFO:teuthology.orchestra.run.smithi092.stdout:958 punch from 0x9ffd600 to 0xa008c00, (0xb600 bytes) 2024-09-15T00:09:15.049 INFO:teuthology.orchestra.run.smithi092.stdout:959 trunc from 0xedcc800 to 0x93f6400 2024-09-15T00:09:15.056 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:15.055+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:15.064 INFO:teuthology.orchestra.run.smithi092.stdout:960 write 0x1afd800 thru 0x1b0ac9b (0xd49c bytes) 2024-09-15T00:09:15.065 INFO:teuthology.orchestra.run.smithi092.stdout:961 read 0x444600 thru 0x448494 (0x3e95 bytes) 2024-09-15T00:09:15.066 INFO:teuthology.orchestra.run.smithi092.stdout:963 read 0x1b0f200 thru 0x1b11bd4 (0x29d5 bytes) 2024-09-15T00:09:15.076 INFO:teuthology.orchestra.run.smithi092.stdout:964 write 0x9bc9000 thru 0x9bd4718 (0xb719 bytes) 2024-09-15T00:09:15.077 INFO:teuthology.orchestra.run.smithi092.stdout:965 read 0x4afe00 thru 0x4bcb70 (0xcd71 bytes) 2024-09-15T00:09:15.078 INFO:teuthology.orchestra.run.smithi092.stdout:966 write 0x1f87a00 thru 0x1f8f531 (0x7b32 bytes) 2024-09-15T00:09:15.079 INFO:teuthology.orchestra.run.smithi092.stdout:969 punch from 0x5e42c00 to 0x5e51200, (0xe600 bytes) 2024-09-15T00:09:15.091 INFO:teuthology.orchestra.run.smithi092.stdout:971 write 0xdeb3200 thru 0xdec319b (0xff9c bytes) 2024-09-15T00:09:15.092 INFO:teuthology.orchestra.run.smithi092.stdout:973 read 0xc372a00 thru 0xc380611 (0xdc12 bytes) 2024-09-15T00:09:15.093 INFO:teuthology.orchestra.run.smithi092.stdout:976 write 0x2b36400 thru 0x2b3b057 (0x4c58 bytes) 2024-09-15T00:09:15.093 INFO:teuthology.orchestra.run.smithi092.stdout:977 read 0xa43d000 thru 0xa44262e (0x562f bytes) 2024-09-15T00:09:15.093 INFO:teuthology.orchestra.run.smithi092.stdout:978 read 0x90e800 thru 0x913d31 (0x5532 bytes) 2024-09-15T00:09:15.099 INFO:teuthology.orchestra.run.smithi092.stdout:979 trunc from 0xdec3200 to 0x9730000 2024-09-15T00:09:15.102 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:15.102 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:15.102 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:15.102 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:15.102 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:15.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:15.119 INFO:teuthology.orchestra.run.smithi092.stdout:980 read 0x65fee00 thru 0x660d633 (0xe834 bytes) 2024-09-15T00:09:15.124 INFO:teuthology.orchestra.run.smithi092.stdout:981 punch from 0x68f8600 to 0x6907c00, (0xf600 bytes) 2024-09-15T00:09:15.124 INFO:teuthology.orchestra.run.smithi092.stdout:986 write 0x25b1600 thru 0x25c0197 (0xeb98 bytes) 2024-09-15T00:09:15.125 INFO:teuthology.orchestra.run.smithi092.stdout:987 write 0xd3000 thru 0xe2732 (0xf733 bytes) 2024-09-15T00:09:15.125 INFO:teuthology.orchestra.run.smithi092.stdout:988 read 0x7c64400 thru 0x7c73709 (0xf30a bytes) 2024-09-15T00:09:15.126 INFO:teuthology.orchestra.run.smithi092.stdout:989 read 0x25f2a00 thru 0x25ff5fb (0xcbfc bytes) 2024-09-15T00:09:15.126 INFO:teuthology.orchestra.run.smithi092.stdout:990 read 0xb83600 thru 0xb92f57 (0xf958 bytes) 2024-09-15T00:09:15.135 INFO:teuthology.orchestra.run.smithi092.stdout:992 trunc from 0x9730000 to 0x25f5c00 2024-09-15T00:09:15.157 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:15.156+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:15.173 INFO:teuthology.orchestra.run.smithi092.stdout:995 write 0x83bfa00 thru 0x83c4c2c (0x522d bytes) 2024-09-15T00:09:15.174 INFO:teuthology.orchestra.run.smithi092.stdout:996 read 0xe9c200 thru 0xeaa571 (0xe372 bytes) 2024-09-15T00:09:15.175 INFO:teuthology.orchestra.run.smithi092.stdout:997 read 0xc9fa00 thru 0xca3e88 (0x4489 bytes) 2024-09-15T00:09:15.175 INFO:teuthology.orchestra.run.smithi092.stdout:1001 write 0x7768e00 thru 0x777000c (0x720d bytes) 2024-09-15T00:09:15.197 INFO:teuthology.orchestra.run.smithi092.stdout:1002 write 0xeb6b400 thru 0xeb77fc0 (0xcbc1 bytes) 2024-09-15T00:09:15.200 INFO:teuthology.orchestra.run.smithi092.stdout:1003 write 0x4a53800 thru 0x4a63409 (0xfc0a bytes) 2024-09-15T00:09:15.211 INFO:teuthology.orchestra.run.smithi092.stdout:1004 trunc from 0xeb78000 to 0x350fe00 2024-09-15T00:09:15.253 INFO:teuthology.orchestra.run.smithi092.stdout:1005 write 0x8373200 thru 0x837eeb1 (0xbcb2 bytes) 2024-09-15T00:09:15.258 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:15.257+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1934 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 557056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.403700+0000", 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8559790429999996, 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011383951999999999, 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.262 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1934 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403700+0000", 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403700+0000", 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403701+0000", 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6100000000000002e-07 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403702+0000", 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7680000000000001e-06 2024-09-15T00:09:15.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403709+0000", 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2419999999999997e-06 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403714+0000", 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.977e-06 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.414929+0000", 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011215289999999999 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.414999+0000", 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0230999999999999e-05 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415084+0000", 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5082999999999998e-05 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1941 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.403720+0000", 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8559596389999999, 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011551284, 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1941 2024-09-15T00:09:15.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403720+0000", 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403720+0000", 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403720+0000", 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5900000000000002e-07 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403723+0000", 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999999e-06 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403729+0000", 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3249999999999996e-06 2024-09-15T00:09:15.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403735+0000", 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1630000000000004e-06 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415112+0000", 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011377073 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415186+0000", 2024-09-15T00:09:15.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4753999999999996e-05 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415271+0000", 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4489999999999999e-05 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1942 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.403742+0000", 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8559371770000004, 2024-09-15T00:09:15.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011710949999999999, 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1942 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403742+0000", 2024-09-15T00:09:15.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403742+0000", 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403743+0000", 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403745+0000", 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9989999999999998e-06 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403751+0000", 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6610000000000003e-06 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403756+0000", 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1939999999999997e-06 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415299+0000", 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011542281999999999 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415370+0000", 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0912000000000001e-05 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415453+0000", 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3535999999999994e-05 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1952 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.403823+0000", 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8558559649999999, 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011844277, 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1952 2024-09-15T00:09:15.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403823+0000", 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403823+0000", 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403824+0000", 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403825+0000", 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7069999999999999e-06 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403832+0000", 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3980000000000004e-06 2024-09-15T00:09:15.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403837+0000", 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9409999999999998e-06 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415493+0000", 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011656631000000001 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415574+0000", 2024-09-15T00:09:15.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415668+0000", 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3702999999999994e-05 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1953 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.403843+0000", 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.855836472, 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012068844, 2024-09-15T00:09:15.280 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1953 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403843+0000", 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403843+0000", 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403843+0000", 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403845+0000", 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8330000000000001e-06 2024-09-15T00:09:15.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403851+0000", 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1859999999999997e-06 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.403857+0000", 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0360000000000003e-06 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415713+0000", 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011855315999999999 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415808+0000", 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5773999999999995e-05 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.415912+0000", 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103319 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2081 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409005+0000", 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8506746490000001, 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011407897, 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2081 2024-09-15T00:09:15.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409005+0000", 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409005+0000", 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409005+0000", 2024-09-15T00:09:15.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409007+0000", 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9539999999999998e-06 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409013+0000", 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2339999999999999e-06 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409018+0000", 2024-09-15T00:09:15.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7360000000000001e-06 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420272+0000", 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011254495999999999 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420338+0000", 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5942e-05 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420413+0000", 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4140999999999994e-05 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2088 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409024+0000", 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8506550209999997, 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01159458, 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.289 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2088 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409024+0000", 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409024+0000", 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409025+0000", 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.96e-07 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409027+0000", 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9860000000000001e-06 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409033+0000", 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.28e-06 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409047+0000", 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3913e-05 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420439+0000", 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011392562 2024-09-15T00:09:15.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420524+0000", 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4586999999999995e-05 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420619+0000", 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4956000000000004e-05 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2089 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409054+0000", 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8506249270000001, 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011788781999999999, 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2089 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409054+0000", 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409054+0000", 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409055+0000", 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8599999999999998e-07 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409057+0000", 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1239999999999999e-06 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409064+0000", 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5590000000000002e-06 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409068+0000", 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420667+0000", 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011598170999999999 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420741+0000", 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4695000000000002e-05 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420843+0000", 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101827 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2101 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.298 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409142+0000", 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8505373069999997, 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011932741, 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2101 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409142+0000", 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409142+0000", 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409142+0000", 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2500000000000001e-07 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409150+0000", 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1550000000000001e-06 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409156+0000", 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6059999999999999e-06 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409161+0000", 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1490000000000002e-06 2024-09-15T00:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420874+0000", 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011712935000000001 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.420962+0000", 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7650000000000003e-05 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421075+0000", 2024-09-15T00:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112821 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2112 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409260+0000", 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8504188609999996, 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012043277, 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2112 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409260+0000", 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409260+0000", 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409261+0000", 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6000000000000004e-07 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409263+0000", 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.136e-06 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409270+0000", 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6059999999999999e-06 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409275+0000", 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2889999999999997e-06 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421118+0000", 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011842831999999999 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421200+0000", 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2568e-05 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421304+0000", 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103286 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2113 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409281+0000", 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8503979900000003, 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012269931, 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2113 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409281+0000", 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409281+0000", 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409282+0000", 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0400000000000002e-07 2024-09-15T00:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409284+0000", 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8920000000000001e-06 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409290+0000", 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.139e-06 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409295+0000", 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421366+0000", 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012071623 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421452+0000", 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5739e-05 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421551+0000", 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9289000000000004e-05 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2120 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409301+0000", 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8503784349999997, 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01246329, 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2120 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409301+0000", 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409301+0000", 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409301+0000", 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2500000000000001e-07 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409303+0000", 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.813e-06 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409309+0000", 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2260000000000002e-06 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409314+0000", 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9690000000000002e-06 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421590+0000", 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012275784 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421677+0000", 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6990000000000006e-05 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421764+0000", 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7182999999999995e-05 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2121 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409320+0000", 2024-09-15T00:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8503590540000001, 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012648379, 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2121 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409320+0000", 2024-09-15T00:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409320+0000", 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409321+0000", 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.53e-07 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409322+0000", 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8139999999999999e-06 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409328+0000", 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0310000000000004e-06 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409333+0000", 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8080000000000004e-06 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421794+0000", 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012460406 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421878+0000", 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4666000000000006e-05 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.421969+0000", 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0301000000000002e-05 2024-09-15T00:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2131 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409408+0000", 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8502711039999999, 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012778451, 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2131 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409408+0000", 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409408+0000", 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409409+0000", 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409415+0000", 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5749999999999997e-06 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409422+0000", 2024-09-15T00:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9059999999999996e-06 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409427+0000", 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2349999999999999e-06 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.422005+0000", 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012578123 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.422082+0000", 2024-09-15T00:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6186000000000005e-05 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.422187+0000", 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105046 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2132 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.409434+0000", 2024-09-15T00:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8502452749999998, 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012908305, 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2132 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409434+0000", 2024-09-15T00:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409434+0000", 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409435+0000", 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409437+0000", 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9980000000000002e-06 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409443+0000", 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5139999999999998e-06 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.409454+0000", 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1195e-05 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.422217+0000", 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012762244000000001 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.422264+0000", 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7314999999999997e-05 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.422342+0000", 2024-09-15T00:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8471000000000001e-05 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2158 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 393216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8473416519999999, 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011263868, 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2158 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412338+0000", 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8099999999999998e-07 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412347+0000", 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6219999999999998e-06 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412354+0000", 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0530000000000001e-06 2024-09-15T00:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412359+0000", 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0479999999999998e-06 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423237+0000", 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010878206 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423499+0000", 2024-09-15T00:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026244000000000001 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423602+0000", 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102118 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2159 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412365+0000", 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8473138890000005, 2024-09-15T00:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011419395000000001, 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2159 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412365+0000", 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412365+0000", 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412366+0000", 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9299999999999999e-07 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412368+0000", 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1430000000000001e-06 2024-09-15T00:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412374+0000", 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2160000000000003e-06 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412390+0000", 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5739e-05 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423644+0000", 2024-09-15T00:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011254504 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423718+0000", 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4041000000000005e-05 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423785+0000", 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6359000000000006e-05 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2170 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412475+0000", 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8472039789999997, 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011441012, 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2170 2024-09-15T00:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412475+0000", 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412475+0000", 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412476+0000", 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8299999999999997e-07 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412478+0000", 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9439999999999999e-06 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412484+0000", 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3099999999999997e-06 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412489+0000", 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4519999999999999e-06 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423814+0000", 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01132536 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423864+0000", 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9781999999999999e-05 2024-09-15T00:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423916+0000", 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2680999999999999e-05 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2178 3.e 3:745ce6a0:::rbd_data.1129feb48f4b.0000000000000002:4 [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412495+0000", 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8471845240000002, 2024-09-15T00:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01159, 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2178 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412495+0000", 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412495+0000", 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412495+0000", 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412499+0000", 2024-09-15T00:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4300000000000002e-06 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412505+0000", 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3210000000000001e-06 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412510+0000", 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6140000000000002e-06 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.423963+0000", 2024-09-15T00:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011453817 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424021+0000", 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7392999999999998e-05 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424085+0000", 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4041999999999994e-05 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:2189 3.e 3:7140b196:::rbd_data.1129feb48f4b.0000000000000009:4 [read 573440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.412583+0000", 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.8470964140000001, 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011684068000000001, 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2189 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412583+0000", 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412583+0000", 2024-09-15T00:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412583+0000", 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412585+0000", 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412592+0000", 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6789999999999999e-06 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.412605+0000", 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2655e-05 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424125+0000", 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011520124 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424204+0000", 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9062999999999998e-05 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.424267+0000", 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3078999999999997e-05 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:15.355 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:1006 write 0xb20ca00 thru 0xb218604 (0xbc05 bytes) 2024-09-15T00:09:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:1009 read 0xaeb3200 thru 0xaeb8b52 (0x5953 bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1010 trunc from 0xb218800 to 0x3a42200 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1012 read 0x2884000 thru 0x288a846 (0x6847 bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1014 punch from 0x1090400 to 0x109ce00, (0xca00 bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1015 punch from 0x1075e00 to 0x107b400, (0x5600 bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1016 read 0x27f1c00 thru 0x2800ce6 (0xf0e7 bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1017 write 0x8898400 thru 0x88992b8 (0xeb9 bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1019 write 0xa4fa00 thru 0xa5cdaf (0xd3b0 bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1021 write 0x6e7a200 thru 0x6e871bc (0xcfbd bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1022 read 0x2e9000 thru 0x2f002c (0x702d bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1023 write 0x5163e00 thru 0x5172fbd (0xf1be bytes) 2024-09-15T00:09:15.356 INFO:teuthology.orchestra.run.smithi092.stdout:1025 clone 9 order 21 su 8192 sc 12 2024-09-15T00:09:15.396 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:15.394+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:15.459 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:15.458+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:15.463 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:15.464 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:15.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:15.560 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:15.559+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:15.578 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:15.579 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:15.587 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:15.671 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:15.670+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:768 3.1f 3:fb2a7c64:::rbd_data.1129aa143e51.0000000000000033:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351183+0000", 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3820635150000005, 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012254846999999999, 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 768 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351183+0000", 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351183+0000", 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351184+0000", 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.08e-07 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351186+0000", 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351192+0000", 2024-09-15T00:09:15.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3799999999999999e-06 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351197+0000", 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0250000000000002e-06 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363234+0000", 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01203688 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363323+0000", 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9169e-05 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363438+0000", 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011500700000000001 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:770 3.1f 3:fb2a7c64:::rbd_data.1129aa143e51.0000000000000033:5 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351204+0000", 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3820433269999999, 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012615664, 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 770 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351204+0000", 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351204+0000", 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351204+0000", 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.39e-07 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351206+0000", 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0669999999999999e-06 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351212+0000", 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9120000000000003e-06 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351217+0000", 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6519999999999997e-06 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363470+0000", 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012253429 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363550+0000", 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0277000000000004e-05 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363819+0000", 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000268888 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:772 3.1f 3:fb2a7c64:::rbd_data.1129aa143e51.0000000000000033:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351223+0000", 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.382024199, 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012778787999999999, 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.745 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 772 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351223+0000", 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351223+0000", 2024-09-15T00:09:15.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351223+0000", 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.72e-07 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351225+0000", 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2110000000000001e-06 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351231+0000", 2024-09-15T00:09:15.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0079999999999999e-06 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351236+0000", 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8890000000000001e-06 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363854+0000", 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012618076000000001 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.363925+0000", 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0976999999999995e-05 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364002+0000", 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6254999999999994e-05 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:817 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351242+0000", 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3820049220000001, 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012974766, 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 817 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351242+0000", 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351242+0000", 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351243+0000", 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0299999999999999e-07 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351245+0000", 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2390000000000001e-06 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351251+0000", 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0090000000000004e-06 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351256+0000", 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6929999999999998e-06 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364032+0000", 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012776556 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364117+0000", 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4902999999999997e-05 2024-09-15T00:09:15.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364217+0000", 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9863000000000003e-05 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:828 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.351262+0000", 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3819846449999993, 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013218841, 2024-09-15T00:09:15.754 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 828 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351262+0000", 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351262+0000", 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351263+0000", 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351271+0000", 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.96e-06 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351277+0000", 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5490000000000003e-06 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.351284+0000", 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:15.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364266+0000", 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012981901000000001 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364358+0000", 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2145000000000006e-05 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.364481+0000", 2024-09-15T00:09:15.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000122953 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1270 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355859+0000", 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3773878289999999, 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012140272000000001, 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1270 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355859+0000", 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355859+0000", 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355860+0000", 2024-09-15T00:09:15.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:09:15.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355862+0000", 2024-09-15T00:09:15.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.216e-06 2024-09-15T00:09:15.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355869+0000", 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.109e-06 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355875+0000", 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5949999999999998e-06 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367642+0000", 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01176779 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367728+0000", 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5809000000000005e-05 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367999+0000", 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027126299999999999 2024-09-15T00:09:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1278 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355882+0000", 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3773651939999993, 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012358408, 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1278 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355882+0000", 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355882+0000", 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355882+0000", 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.39e-07 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355884+0000", 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.995e-06 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355891+0000", 2024-09-15T00:09:15.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9870000000000002e-06 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355897+0000", 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5099999999999998e-06 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368049+0000", 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012152586999999999 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368136+0000", 2024-09-15T00:09:15.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6255999999999995e-05 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368240+0000", 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000104634 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1279 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355903+0000", 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3773438139999996, 2024-09-15T00:09:15.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012602021999999999, 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1279 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355903+0000", 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355903+0000", 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355904+0000", 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355906+0000", 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9240000000000001e-06 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355913+0000", 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.029e-06 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355918+0000", 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6080000000000003e-06 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368287+0000", 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012368580000000001 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368382+0000", 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5455e-05 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368505+0000", 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012301599999999999 2024-09-15T00:09:15.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1287 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355925+0000", 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3773221539999998, 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012827722999999999, 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1287 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355925+0000", 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355925+0000", 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355925+0000", 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355927+0000", 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9530000000000002e-06 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355934+0000", 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5520000000000001e-06 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355939+0000", 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4369999999999998e-06 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368557+0000", 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012617712999999999 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368643+0000", 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6011e-05 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368753+0000", 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109638 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1288 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355946+0000", 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3773009050000002, 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013021272, 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1288 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355946+0000", 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355946+0000", 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355947+0000", 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3099999999999998e-07 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355949+0000", 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1579999999999999e-06 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355961+0000", 2024-09-15T00:09:15.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1901e-05 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355967+0000", 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9399999999999999e-06 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368787+0000", 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012820249000000001 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368868+0000", 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1384999999999994e-05 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368967+0000", 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9208000000000003e-05 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1295 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.781 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.355974+0000", 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3772727719999995, 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013206915, 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1295 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355974+0000", 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355974+0000", 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355975+0000", 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6800000000000001e-07 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355977+0000", 2024-09-15T00:09:15.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1320000000000001e-06 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355984+0000", 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8650000000000003e-06 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.355990+0000", 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8839999999999999e-06 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368999+0000", 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013009121 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369080+0000", 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1038000000000006e-05 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369181+0000", 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101407 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.785 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1296 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 401408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3772469750000003, 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013357545, 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1296 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356001+0000", 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356003+0000", 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.486e-06 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356010+0000", 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3459999999999996e-06 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356016+0000", 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4990000000000002e-06 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369219+0000", 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013202894999999999 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369291+0000", 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2458000000000003e-05 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369358+0000", 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6370000000000003e-05 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1325 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 294912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.790 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356044+0000", 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3772026890000006, 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013560946000000001, 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1325 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.791 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356044+0000", 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356044+0000", 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356045+0000", 2024-09-15T00:09:15.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2300000000000002e-07 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356048+0000", 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5729999999999999e-06 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356057+0000", 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0440000000000001e-06 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356063+0000", 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8619999999999999e-06 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369423+0000", 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013360276000000001 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369513+0000", 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9617000000000006e-05 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369605+0000", 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2151000000000006e-05 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1326 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356070+0000", 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3771765340000002, 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013717217, 2024-09-15T00:09:15.795 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1326 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356070+0000", 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356070+0000", 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356071+0000", 2024-09-15T00:09:15.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6199999999999998e-07 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356073+0000", 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0640000000000001e-06 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356080+0000", 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7329999999999997e-06 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356085+0000", 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0649999999999998e-06 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369652+0000", 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013567106000000001 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369722+0000", 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9794000000000003e-05 2024-09-15T00:09:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369788+0000", 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5993000000000003e-05 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1334 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 491520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356091+0000", 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.377155514, 2024-09-15T00:09:15.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013921717, 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1334 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356091+0000", 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356091+0000", 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356092+0000", 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.27e-07 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356094+0000", 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0530000000000001e-06 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356101+0000", 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6460000000000004e-06 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356106+0000", 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1880000000000001e-06 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369824+0000", 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013718340000000001 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369905+0000", 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0694999999999999e-05 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370013+0000", 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108368 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.804 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1335 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 409600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356112+0000", 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3771347459999994, 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014060968, 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1335 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356112+0000", 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356112+0000", 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:15.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7800000000000002e-07 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356124+0000", 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1223000000000001e-05 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356131+0000", 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1860000000000003e-06 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356138+0000", 2024-09-15T00:09:15.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2089999999999999e-06 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370081+0000", 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01394226 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370124+0000", 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3551999999999999e-05 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370173+0000", 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9060000000000001e-05 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1364 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356168+0000", 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3770792570000001, 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014233545, 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1364 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356168+0000", 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356168+0000", 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356168+0000", 2024-09-15T00:09:15.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:15.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:15.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.046e-06 2024-09-15T00:09:15.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356177+0000", 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5039999999999999e-06 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356182+0000", 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8659999999999997e-06 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370209+0000", 2024-09-15T00:09:15.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014027181 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370284+0000", 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.525e-05 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370401+0000", 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011728800000000001 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1365 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356189+0000", 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3770583809999994, 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014419042999999999, 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1365 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356189+0000", 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356189+0000", 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356189+0000", 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4999999999999998e-07 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356191+0000", 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356198+0000", 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7730000000000002e-06 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356204+0000", 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5890000000000002e-06 2024-09-15T00:09:15.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370434+0000", 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014229980999999999 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370508+0000", 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4011999999999995e-05 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370608+0000", 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100058 2024-09-15T00:09:15.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1373 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356210+0000", 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.377037391, 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0145708, 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1373 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356210+0000", 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356210+0000", 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356210+0000", 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.65e-07 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356212+0000", 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.858e-06 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356218+0000", 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5710000000000003e-06 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356224+0000", 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2079999999999999e-06 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370640+0000", 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014416698 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370693+0000", 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2326e-05 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370780+0000", 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7774000000000004e-05 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1374 3.1f 3:f86c7358:::rbd_data.1129feb48f4b.0000000000000014:4 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:15.823 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.3770170289999992, 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014785247, 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1374 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:15.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.01e-07 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356232+0000", 2024-09-15T00:09:15.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0269999999999998e-06 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356239+0000", 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4030000000000004e-06 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.356244+0000", 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8749999999999999e-06 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:15.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370813+0000", 2024-09-15T00:09:15.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014569248999999999 2024-09-15T00:09:15.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:15.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370913+0000", 2024-09-15T00:09:15.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100144 2024-09-15T00:09:15.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:15.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:15.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:15.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371015+0000", 2024-09-15T00:09:15.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102148 2024-09-15T00:09:15.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.830 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:15.830 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:15.830 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:15.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:15.832 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:15.829+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:15.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:15.931+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:15.981 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:15.981 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:15.981 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:15.981 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:15.990 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:16.074 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:16.073+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:16.134 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:16.133+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:16.142 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:16.142 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:16.142 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:16.142 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:16.143 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:16.151 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:16.207 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-15T00:09:16.235 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:16.234+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:16.310 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1237 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 401408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353675+0000", 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9544860459999995, 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013309087000000001, 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1237 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353675+0000", 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353675+0000", 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353676+0000", 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.34e-07 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353678+0000", 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.99e-06 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353684+0000", 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0909999999999998e-06 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353689+0000", 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8760000000000004e-06 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.366810+0000", 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013120672999999999 2024-09-15T00:09:16.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.366889+0000", 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9314999999999995e-05 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.366985+0000", 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5707999999999999e-05 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.315 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1238 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353694+0000", 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9544670859999993, 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013468529, 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1238 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353694+0000", 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353694+0000", 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353695+0000", 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353697+0000", 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.764e-06 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353702+0000", 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7649999999999999e-06 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353708+0000", 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1419999999999998e-06 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367014+0000", 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013305895999999999 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367082+0000", 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7415000000000004e-05 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367163+0000", 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1170999999999999e-05 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1239 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353715+0000", 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9544460800000003, 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013586938999999999, 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1239 2024-09-15T00:09:16.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353715+0000", 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353715+0000", 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353716+0000", 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353718+0000", 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.897e-06 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353724+0000", 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8250000000000001e-06 2024-09-15T00:09:16.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353728+0000", 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5589999999999999e-06 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367198+0000", 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013470058 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367246+0000", 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8013000000000003e-05 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367302+0000", 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6239999999999997e-05 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1240 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.324 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9544269980000006, 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013711031, 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1240 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:16.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:16.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:16.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:09:16.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353737+0000", 2024-09-15T00:09:16.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.844e-06 2024-09-15T00:09:16.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353744+0000", 2024-09-15T00:09:16.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0310000000000001e-06 2024-09-15T00:09:16.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353748+0000", 2024-09-15T00:09:16.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6569999999999996e-06 2024-09-15T00:09:16.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367332+0000", 2024-09-15T00:09:16.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013583556 2024-09-15T00:09:16.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367368+0000", 2024-09-15T00:09:16.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5796000000000001e-05 2024-09-15T00:09:16.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367446+0000", 2024-09-15T00:09:16.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7792000000000003e-05 2024-09-15T00:09:16.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.332 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.333 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1241 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.333 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353754+0000", 2024-09-15T00:09:16.333 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9544073609999995, 2024-09-15T00:09:16.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013907709000000001, 2024-09-15T00:09:16.334 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.334 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.334 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.334 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1241 2024-09-15T00:09:16.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.334 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353754+0000", 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353754+0000", 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:16.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6100000000000002e-07 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353756+0000", 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.818e-06 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353764+0000", 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.435e-06 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353768+0000", 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6970000000000001e-06 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367483+0000", 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013714835999999999 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367559+0000", 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6106000000000006e-05 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367662+0000", 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102456 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1242 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353774+0000", 2024-09-15T00:09:16.338 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9543871829999997, 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014339533, 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1242 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353774+0000", 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353774+0000", 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353775+0000", 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.39e-07 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353777+0000", 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8950000000000001e-06 2024-09-15T00:09:16.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353782+0000", 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8479999999999997e-06 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353801+0000", 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8209000000000001e-05 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367715+0000", 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013914358 2024-09-15T00:09:16.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367811+0000", 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5613999999999996e-05 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368114+0000", 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030327000000000002 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1244 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9543284819999993, 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014385056, 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1244 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.343 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353836+0000", 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5670000000000002e-06 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353845+0000", 2024-09-15T00:09:16.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7630000000000006e-06 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353850+0000", 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2850000000000002e-06 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368136+0000", 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014285791000000001 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368174+0000", 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8414999999999997e-05 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368218+0000", 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3791999999999997e-05 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.347 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1245 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353858+0000", 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9543032839999999, 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014529564999999999, 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1245 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.348 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353858+0000", 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353858+0000", 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353859+0000", 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9800000000000004e-07 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353861+0000", 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0059999999999999e-06 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353867+0000", 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9780000000000002e-06 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353871+0000", 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6779999999999999e-06 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368248+0000", 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014376172 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368313+0000", 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5597000000000002e-05 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368388+0000", 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4635999999999995e-05 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1264 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9542379380000003, 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014693985, 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1264 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.353 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6499999999999999e-07 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353927+0000", 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1489999999999998e-06 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353934+0000", 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0299999999999996e-06 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353940+0000", 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5759999999999996e-06 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368429+0000", 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014489007999999999 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368517+0000", 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8008000000000002e-05 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368618+0000", 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010074899999999999 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1265 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9542127740000002, 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01488623, 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1265 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353951+0000", 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:09:16.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353958+0000", 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2620000000000004e-06 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353963+0000", 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9819999999999999e-06 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368649+0000", 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014686716000000001 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368734+0000", 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4893000000000002e-05 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368835+0000", 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010072 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1271 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9541454189999996, 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015034792999999999, 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1271 2024-09-15T00:09:16.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354018+0000", 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8059999999999999e-06 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354024+0000", 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8540000000000002e-06 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354029+0000", 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6550000000000003e-06 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368868+0000", 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014839395999999999 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368954+0000", 2024-09-15T00:09:16.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5636999999999993e-05 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369051+0000", 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7095999999999993e-05 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1272 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354034+0000", 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9541271150000004, 2024-09-15T00:09:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015239717, 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1272 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354034+0000", 2024-09-15T00:09:16.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354034+0000", 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354035+0000", 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354037+0000", 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.849e-06 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354042+0000", 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7250000000000002e-06 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354047+0000", 2024-09-15T00:09:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369095+0000", 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015047955999999999 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369177+0000", 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2212000000000005e-05 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369274+0000", 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6811999999999994e-05 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1280 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354092+0000", 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.954069252, 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01540946, 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1280 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354092+0000", 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354092+0000", 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354093+0000", 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354094+0000", 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.824e-06 2024-09-15T00:09:16.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354100+0000", 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8259999999999998e-06 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354105+0000", 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4309999999999996e-06 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369307+0000", 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015202466 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369401+0000", 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4327999999999995e-05 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369502+0000", 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100243 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.375 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1289 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354154+0000", 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9540071470000004, 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015611507, 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1289 2024-09-15T00:09:16.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354154+0000", 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354154+0000", 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354155+0000", 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8500000000000002e-07 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354157+0000", 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.26e-06 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354163+0000", 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8220000000000003e-06 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354168+0000", 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8899999999999998e-06 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369559+0000", 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015391103999999999 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369656+0000", 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6877000000000001e-05 2024-09-15T00:09:16.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369766+0000", 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110169 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1316 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9539690709999995, 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015835378000000001, 2024-09-15T00:09:16.380 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1316 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354193+0000", 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354195+0000", 2024-09-15T00:09:16.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.77e-06 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354200+0000", 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6670000000000001e-06 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354205+0000", 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.78e-06 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369836+0000", 2024-09-15T00:09:16.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015630498 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369924+0000", 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8707000000000003e-05 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370028+0000", 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010359 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1317 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9539507910000005, 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016038510999999998, 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1317 2024-09-15T00:09:16.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354219+0000", 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3310000000000004e-06 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354227+0000", 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354232+0000", 2024-09-15T00:09:16.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.758e-06 2024-09-15T00:09:16.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370061+0000", 2024-09-15T00:09:16.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015829006 2024-09-15T00:09:16.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370151+0000", 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9575999999999998e-05 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370249+0000", 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8743e-05 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.389 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1327 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9538608469999996, 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016224237999999998, 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1327 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354303+0000", 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.277e-06 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354310+0000", 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2450000000000003e-06 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354315+0000", 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0799999999999996e-06 2024-09-15T00:09:16.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370290+0000", 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015975552000000001 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370413+0000", 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012316399999999999 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370525+0000", 2024-09-15T00:09:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000111537 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1355 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9538048729999993, 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016460124999999999, 2024-09-15T00:09:16.394 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1355 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:16.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:16.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354359+0000", 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0590000000000001e-06 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354365+0000", 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7769999999999999e-06 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354370+0000", 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370550+0000", 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016180486000000001 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370595+0000", 2024-09-15T00:09:16.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5055999999999997e-05 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370817+0000", 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022162199999999999 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1356 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9537865449999998, 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016655262000000001, 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1356 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5600000000000001e-07 2024-09-15T00:09:16.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354377+0000", 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7710000000000001e-06 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354383+0000", 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8429999999999998e-06 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354388+0000", 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.583e-06 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370848+0000", 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016460174000000001 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370934+0000", 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6431e-05 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371030+0000", 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6003999999999998e-05 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1366 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.9536959649999996, 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016710004000000001, 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:16.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1366 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:16.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354468+0000", 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.159e-06 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354474+0000", 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2720000000000003e-06 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354479+0000", 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8420000000000004e-06 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371054+0000", 2024-09-15T00:09:16.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016575185999999999 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371110+0000", 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5356999999999999e-05 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371176+0000", 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5801000000000002e-05 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:16.409 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:16.409 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:16.410 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:16.411 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:16.409+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:16.511 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:16.509+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:16.569 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:16.569 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:16.569 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:16.569 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:16.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:16.612 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:16.611+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:16.699 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:16.699 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:16.699 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:16.700 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:16.700 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:16.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:16.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:16.750 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:16.749+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:16.814 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:16.813+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:16.827 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:16.845 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:16.845 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:16.858 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:09:16.862 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:16.862 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:16.862 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:16.862 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:16.863 INFO:teuthology.orchestra.run.smithi092.stdout:1027 trunc from 0x8899400 to 0x663ce00 2024-09-15T00:09:16.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:16.890 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:16.913 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:16.916 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:16.915+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:16.938 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:16.939 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:16.971 INFO:teuthology.orchestra.run.smithi092.stdout:1028 read 0x5f58800 thru 0x5f62665 (0x9e66 bytes) 2024-09-15T00:09:16.996 INFO:teuthology.orchestra.run.smithi092.stdout:1030 write 0xd90b600 thru 0xd91b3c5 (0xfdc6 bytes) 2024-09-15T00:09:16.997 INFO:teuthology.orchestra.run.smithi092.stdout:1031 write 0x69a4a00 thru 0x69abc7b (0x727c bytes) 2024-09-15T00:09:16.997 INFO:teuthology.orchestra.run.smithi092.stdout:1032 punch from 0x8318200 to 0x8325400, (0xd200 bytes) 2024-09-15T00:09:17.002 INFO:teuthology.orchestra.run.smithi092.stdout:1033 punch from 0x3346e00 to 0x334a200, (0x3400 bytes) 2024-09-15T00:09:17.006 INFO:teuthology.orchestra.run.smithi092.stdout:1037 write 0x705f000 thru 0x706c014 (0xd015 bytes) 2024-09-15T00:09:17.006 INFO:teuthology.orchestra.run.smithi092.stdout:1038 write 0x5247200 thru 0x5250f94 (0x9d95 bytes) 2024-09-15T00:09:17.016 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:17.015+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:17.019 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:17.019 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:17.019 INFO:teuthology.orchestra.run.smithi092.stdout:1039 read 0x729aa00 thru 0x72a8463 (0xda64 bytes) 2024-09-15T00:09:17.026 INFO:teuthology.orchestra.run.smithi092.stdout:1042 punch from 0xb472000 to 0xb47b200, (0x9200 bytes) 2024-09-15T00:09:17.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:17.040 INFO:teuthology.orchestra.run.smithi092.stdout:1045 trunc from 0xd91b400 to 0x8ab9000 2024-09-15T00:09:17.117 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:17.116+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:17.130 INFO:teuthology.orchestra.run.smithi092.stdout:1046 punch from 0x5c11400 to 0x5c1d400, (0xc000 bytes) 2024-09-15T00:09:17.193 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:17.193 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:17.201 INFO:teuthology.orchestra.run.smithi092.stdout:1047 write 0xcbc5a00 thru 0xcbccc94 (0x7295 bytes) 2024-09-15T00:09:17.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:17.208 INFO:teuthology.orchestra.run.smithi092.stdout:1048 write 0x63b3c00 thru 0x63c2406 (0xe807 bytes) 2024-09-15T00:09:17.209 INFO:teuthology.orchestra.run.smithi092.stdout:1049 trunc from 0xcbcce00 to 0xc5b0c00 2024-09-15T00:09:17.219 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:17.218+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:17.229 INFO:teuthology.orchestra.run.smithi092.stdout:1052 read 0x11f3800 thru 0x11f401b (0x81c bytes) 2024-09-15T00:09:17.230 INFO:teuthology.orchestra.run.smithi092.stdout:1053 punch from 0x3501400 to 0x350d200, (0xbe00 bytes) 2024-09-15T00:09:17.245 INFO:teuthology.orchestra.run.smithi092.stdout:1054 write 0x69fce00 thru 0x6a068f8 (0x9af9 bytes) 2024-09-15T00:09:17.300 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xee2e800 2024-09-15T00:09:17.303 INFO:teuthology.orchestra.run.smithi092.stdout:1055 trunc from 0xc5b0c00 to 0xee2e800 2024-09-15T00:09:17.320 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:17.319+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:17.343 INFO:teuthology.orchestra.run.smithi092.stdout:1056 write 0x4ef3800 thru 0x4ef4a88 (0x1289 bytes) 2024-09-15T00:09:17.344 INFO:teuthology.orchestra.run.smithi092.stdout:1060 write 0x385a00 thru 0x387a59 (0x205a bytes) 2024-09-15T00:09:17.345 INFO:teuthology.orchestra.run.smithi092.stdout:1061 write 0x9aafc00 thru 0x9abfbb1 (0xffb2 bytes) 2024-09-15T00:09:17.345 INFO:teuthology.orchestra.run.smithi092.stdout:1062 punch from 0x4b6c400 to 0x4b78800, (0xc400 bytes) 2024-09-15T00:09:17.349 INFO:teuthology.orchestra.run.smithi092.stdout:1063 read 0x290a000 thru 0x2913477 (0x9478 bytes) 2024-09-15T00:09:17.360 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:17.360 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:17.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:17.381 INFO:teuthology.orchestra.run.smithi092.stdout:1064 punch from 0xc68a00 to 0xc69a00, (0x1000 bytes) 2024-09-15T00:09:17.383 INFO:teuthology.orchestra.run.smithi092.stdout:1066 write 0x603aa00 thru 0x603dad5 (0x30d6 bytes) 2024-09-15T00:09:17.396 INFO:teuthology.orchestra.run.smithi092.stdout:1072 write 0x80a8e00 thru 0x80ac074 (0x3275 bytes) 2024-09-15T00:09:17.415 INFO:teuthology.orchestra.run.smithi092.stdout:1073 trunc from 0xee2e800 to 0x1f0e00 2024-09-15T00:09:17.420 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:17.419+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:17.522 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:17.520+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:17.526 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:17.526 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:17.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:17.622 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:17.621+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:17.623 INFO:teuthology.orchestra.run.smithi092.stdout:1074 write 0x5aee000 thru 0x5af9c2e (0xbc2f bytes) 2024-09-15T00:09:17.623 INFO:teuthology.orchestra.run.smithi092.stdout:1080 read 0x19cb200 thru 0x19d38c5 (0x86c6 bytes) 2024-09-15T00:09:17.623 INFO:teuthology.orchestra.run.smithi092.stdout:1082 punch from 0x5290600 to 0x5294000, (0x3a00 bytes) 2024-09-15T00:09:17.625 INFO:teuthology.orchestra.run.smithi092.stdout:1084 read 0x369bc00 thru 0x369cb47 (0xf48 bytes) 2024-09-15T00:09:17.625 INFO:teuthology.orchestra.run.smithi092.stdout:1086 read 0x1ccbc00 thru 0x1cd73c3 (0xb7c4 bytes) 2024-09-15T00:09:17.625 INFO:teuthology.orchestra.run.smithi092.stdout:1088 punch from 0x423e600 to 0x424b200, (0xcc00 bytes) 2024-09-15T00:09:17.625 INFO:teuthology.orchestra.run.smithi092.stdout:1089 punch from 0x18ac00 to 0x18ba00, (0xe00 bytes) 2024-09-15T00:09:17.625 INFO:teuthology.orchestra.run.smithi092.stdout:1090 punch from 0x57e5200 to 0x57e8200, (0x3000 bytes) 2024-09-15T00:09:17.625 INFO:teuthology.orchestra.run.smithi092.stdout:1091 read 0x1524a00 thru 0x1532b71 (0xe172 bytes) 2024-09-15T00:09:17.648 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:17.648 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:17.654 INFO:teuthology.orchestra.run.smithi092.stdout:1092 write 0xa56b400 thru 0xa57a117 (0xed18 bytes) 2024-09-15T00:09:17.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:17.679 INFO:teuthology.orchestra.run.smithi092.stdout:1093 write 0xda85400 thru 0xda8ce75 (0x7a76 bytes) 2024-09-15T00:09:17.680 INFO:teuthology.orchestra.run.smithi092.stdout:1094 punch from 0xbbf4200 to 0xbbf5e00, (0x1c00 bytes) 2024-09-15T00:09:17.694 INFO:teuthology.orchestra.run.smithi092.stdout:1096 trunc from 0xda8d000 to 0x1b3de00 2024-09-15T00:09:17.735 INFO:teuthology.orchestra.run.smithi092.stdout:1098 write 0x4186e00 thru 0x418ad65 (0x3f66 bytes) 2024-09-15T00:09:17.736 INFO:teuthology.orchestra.run.smithi092.stdout:1101 punch from 0x26b6000 to 0x26bf800, (0x9800 bytes) 2024-09-15T00:09:17.737 INFO:teuthology.orchestra.run.smithi092.stdout:1102 punch from 0x2db2c00 to 0x2dbe800, (0xbc00 bytes) 2024-09-15T00:09:17.737 INFO:teuthology.orchestra.run.smithi092.stdout:1104 read 0xc59800 thru 0xc60d2e (0x752f bytes) 2024-09-15T00:09:17.740 INFO:teuthology.orchestra.run.smithi092.stdout:1109 punch from 0x877e00 to 0x884a00, (0xcc00 bytes) 2024-09-15T00:09:17.747 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:17.746+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:17.763 INFO:teuthology.orchestra.run.smithi092.stdout:1110 write 0xe2fcc00 thru 0xe2fcddd (0x1de bytes) 2024-09-15T00:09:17.764 INFO:teuthology.orchestra.run.smithi092.stdout:1114 read 0x216ce00 thru 0x216f077 (0x2278 bytes) 2024-09-15T00:09:17.768 INFO:teuthology.orchestra.run.smithi092.stdout:1116 write 0xa626400 thru 0xa63045d (0xa05e bytes) 2024-09-15T00:09:17.784 INFO:teuthology.orchestra.run.smithi092.stdout:1119 trunc from 0xe2fce00 to 0xc94600 2024-09-15T00:09:17.822 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:17.822 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:17.823 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:17.822+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:17.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:17.833 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-15T00:09:17.833 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:09:17.833 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-15T00:09:17.833 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T00:09:17.833 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T00:09:17.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-15T00:09:17.841 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:09:17.859 INFO:teuthology.orchestra.run.smithi092.stdout:1121 write 0x8b0c000 thru 0x8b1790b (0xb90c bytes) 2024-09-15T00:09:17.880 INFO:teuthology.orchestra.run.smithi092.stdout:1123 write 0xb797400 thru 0xb7a53d2 (0xdfd3 bytes) 2024-09-15T00:09:17.903 INFO:teuthology.orchestra.run.smithi092.stdout:1126 write 0xbe15400 thru 0xbe2055d (0xb15e bytes) 2024-09-15T00:09:17.904 INFO:teuthology.orchestra.run.smithi092.stdout:1127 write 0x461b400 thru 0x462620d (0xae0e bytes) 2024-09-15T00:09:17.904 INFO:teuthology.orchestra.run.smithi092.stdout:1128 write 0x643f600 thru 0x6440878 (0x1279 bytes) 2024-09-15T00:09:17.904 INFO:teuthology.orchestra.run.smithi092.stdout:1129 read 0x9f41e00 thru 0x9f4414d (0x234e bytes) 2024-09-15T00:09:17.904 INFO:teuthology.orchestra.run.smithi092.stdout:1130 write 0x1485a00 thru 0x1488a5d (0x305e bytes) 2024-09-15T00:09:17.905 INFO:teuthology.orchestra.run.smithi092.stdout:1131 write 0x9942400 thru 0x99435f9 (0x11fa bytes) 2024-09-15T00:09:17.905 INFO:teuthology.orchestra.run.smithi092.stdout:1136 read 0x4a04e00 thru 0x4a0657d (0x177e bytes) 2024-09-15T00:09:17.905 INFO:teuthology.orchestra.run.smithi092.stdout:1137 punch from 0x58a4a00 to 0x58ab600, (0x6c00 bytes) 2024-09-15T00:09:17.905 INFO:teuthology.orchestra.run.smithi092.stdout:1139 read 0x19e3e00 thru 0x19e687b (0x2a7c bytes) 2024-09-15T00:09:17.905 INFO:teuthology.orchestra.run.smithi092.stdout:1140 clone 10 order 18 su 4096 sc 12 2024-09-15T00:09:17.925 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:17.924+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T00:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:17.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:18.039 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:18.037+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:18.126 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:18.125+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:18.141 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:18.141 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:18.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:18.227 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:18.226+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:18.292 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:18.292 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:18.301 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:18.345 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:18.344+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:18.422 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:18.422 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:18.430 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:18.428+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:18.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:18.530 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:18.529+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:18.581 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:18.582 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:18.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:18.631 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:18.630+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:18.732 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:18.730+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:18.737 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:18.737 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:18.747 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:18.833 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:18.831+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:18.861 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:18.862 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:18.869 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:18.915 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone9 from 0xbe20600 (overlap 0x1f0e00) to 0x12a200 2024-09-15T00:09:18.934 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:18.932+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:18.956 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:18.988 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:18.988 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:18.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:19.034 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:19.033+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:19.089 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:19.089 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:19.106 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:19.106 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:19.123 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:19.123 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:19.135 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:19.134+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:19.140 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:19.141 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:19.157 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:19.157 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:19.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:19.157 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:19.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.157 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1237 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 401408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.157 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353675+0000", 2024-09-15T00:09:19.157 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800821166, 2024-09-15T00:09:19.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013309087000000001, 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1237 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353675+0000", 2024-09-15T00:09:19.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353675+0000", 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353676+0000", 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.34e-07 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353678+0000", 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.99e-06 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353684+0000", 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0909999999999998e-06 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353689+0000", 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8760000000000004e-06 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.366810+0000", 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013120672999999999 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.366889+0000", 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9314999999999995e-05 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.366985+0000", 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5707999999999999e-05 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1238 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353694+0000", 2024-09-15T00:09:19.162 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800802206, 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013468529, 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1238 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353694+0000", 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353694+0000", 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353695+0000", 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:09:19.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353697+0000", 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.764e-06 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353702+0000", 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7649999999999999e-06 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353708+0000", 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1419999999999998e-06 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367014+0000", 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013305895999999999 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367082+0000", 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7415000000000004e-05 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367163+0000", 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1170999999999999e-05 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1239 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353715+0000", 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800781199999999, 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013586938999999999, 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.167 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1239 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353715+0000", 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353715+0000", 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353716+0000", 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353718+0000", 2024-09-15T00:09:19.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.897e-06 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353724+0000", 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8250000000000001e-06 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353728+0000", 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5589999999999999e-06 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367198+0000", 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013470058 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367246+0000", 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8013000000000003e-05 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367302+0000", 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6239999999999997e-05 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.171 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1240 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800762118, 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013711031, 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.172 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1240 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:19.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353735+0000", 2024-09-15T00:09:19.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:09:19.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353737+0000", 2024-09-15T00:09:19.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.844e-06 2024-09-15T00:09:19.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353744+0000", 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0310000000000001e-06 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353748+0000", 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6569999999999996e-06 2024-09-15T00:09:19.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367332+0000", 2024-09-15T00:09:19.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013583556 2024-09-15T00:09:19.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367368+0000", 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5796000000000001e-05 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367446+0000", 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7792000000000003e-05 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1241 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353754+0000", 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800742481, 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013907709000000001, 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1241 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353754+0000", 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353754+0000", 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6100000000000002e-07 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353756+0000", 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.818e-06 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353764+0000", 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.435e-06 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353768+0000", 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6970000000000001e-06 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367483+0000", 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013714835999999999 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367559+0000", 2024-09-15T00:09:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6106000000000006e-05 2024-09-15T00:09:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367662+0000", 2024-09-15T00:09:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102456 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1242 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353774+0000", 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800722303000001, 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014339533, 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1242 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353774+0000", 2024-09-15T00:09:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353774+0000", 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353775+0000", 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.39e-07 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353777+0000", 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8950000000000001e-06 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353782+0000", 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8479999999999997e-06 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353801+0000", 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8209000000000001e-05 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367715+0000", 2024-09-15T00:09:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.013914358 2024-09-15T00:09:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.367811+0000", 2024-09-15T00:09:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5613999999999996e-05 2024-09-15T00:09:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368114+0000", 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030327000000000002 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1244 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:19.189 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800663602, 2024-09-15T00:09:19.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014385056, 2024-09-15T00:09:19.190 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.190 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.190 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.191 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1244 2024-09-15T00:09:19.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.191 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:19.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:19.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353833+0000", 2024-09-15T00:09:19.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:09:19.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353836+0000", 2024-09-15T00:09:19.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5670000000000002e-06 2024-09-15T00:09:19.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353845+0000", 2024-09-15T00:09:19.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7630000000000006e-06 2024-09-15T00:09:19.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353850+0000", 2024-09-15T00:09:19.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2850000000000002e-06 2024-09-15T00:09:19.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368136+0000", 2024-09-15T00:09:19.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014285791000000001 2024-09-15T00:09:19.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368174+0000", 2024-09-15T00:09:19.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8414999999999997e-05 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368218+0000", 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3791999999999997e-05 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1245 3.b 3:d136f54c:::rbd_data.1129aa143e51.0000000000000036:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.199 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353858+0000", 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800638404000001, 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014529564999999999, 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1245 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353858+0000", 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353858+0000", 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353859+0000", 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9800000000000004e-07 2024-09-15T00:09:19.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353861+0000", 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0059999999999999e-06 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353867+0000", 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9780000000000002e-06 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353871+0000", 2024-09-15T00:09:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6779999999999999e-06 2024-09-15T00:09:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368248+0000", 2024-09-15T00:09:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014376172 2024-09-15T00:09:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368313+0000", 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5597000000000002e-05 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368388+0000", 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4635999999999995e-05 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1264 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800573057999999, 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014693985, 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1264 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353924+0000", 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6499999999999999e-07 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353927+0000", 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1489999999999998e-06 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353934+0000", 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0299999999999996e-06 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353940+0000", 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5759999999999996e-06 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368429+0000", 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014489007999999999 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368517+0000", 2024-09-15T00:09:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8008000000000002e-05 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368618+0000", 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010074899999999999 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1265 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800547893999999, 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01488623, 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1265 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353949+0000", 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353951+0000", 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2369999999999999e-06 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353958+0000", 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2620000000000004e-06 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353963+0000", 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9819999999999999e-06 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368649+0000", 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014686716000000001 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368734+0000", 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4893000000000002e-05 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368835+0000", 2024-09-15T00:09:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010072 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1271 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800480539, 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015034792999999999, 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.214 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1271 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354016+0000", 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354018+0000", 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8059999999999999e-06 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354024+0000", 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8540000000000002e-06 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354029+0000", 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6550000000000003e-06 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368868+0000", 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.014839395999999999 2024-09-15T00:09:19.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.368954+0000", 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5636999999999993e-05 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369051+0000", 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7095999999999993e-05 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1272 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354034+0000", 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800462234999999, 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015239717, 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1272 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.219 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354034+0000", 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354034+0000", 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354035+0000", 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354037+0000", 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.849e-06 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354042+0000", 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7250000000000002e-06 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354047+0000", 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369095+0000", 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015047955999999999 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369177+0000", 2024-09-15T00:09:19.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2212000000000005e-05 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369274+0000", 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6811999999999994e-05 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1280 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354092+0000", 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800404371999999, 2024-09-15T00:09:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01540946, 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1280 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354092+0000", 2024-09-15T00:09:19.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354092+0000", 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354093+0000", 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354094+0000", 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.824e-06 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354100+0000", 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8259999999999998e-06 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354105+0000", 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4309999999999996e-06 2024-09-15T00:09:19.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369307+0000", 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015202466 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369401+0000", 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4327999999999995e-05 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369502+0000", 2024-09-15T00:09:19.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100243 2024-09-15T00:09:19.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.228 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.229 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1289 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.229 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354154+0000", 2024-09-15T00:09:19.229 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800342267, 2024-09-15T00:09:19.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015611507, 2024-09-15T00:09:19.230 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.232 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.232 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.232 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1289 2024-09-15T00:09:19.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.232 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354154+0000", 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354154+0000", 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354155+0000", 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8500000000000002e-07 2024-09-15T00:09:19.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354157+0000", 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.26e-06 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354163+0000", 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8220000000000003e-06 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354168+0000", 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8899999999999998e-06 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369559+0000", 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015391103999999999 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369656+0000", 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6877000000000001e-05 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369766+0000", 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110169 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1316 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800304191, 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015835378000000001, 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1316 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354193+0000", 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354195+0000", 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.77e-06 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354200+0000", 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6670000000000001e-06 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354205+0000", 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.78e-06 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369836+0000", 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015630498 2024-09-15T00:09:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.369924+0000", 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8707000000000003e-05 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370028+0000", 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010359 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1317 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800285911, 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016038510999999998, 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1317 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354211+0000", 2024-09-15T00:09:19.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354219+0000", 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3310000000000004e-06 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354227+0000", 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354232+0000", 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.758e-06 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370061+0000", 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015829006 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370151+0000", 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9575999999999998e-05 2024-09-15T00:09:19.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370249+0000", 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8743e-05 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1327 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800195967000001, 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016224237999999998, 2024-09-15T00:09:19.245 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1327 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354301+0000", 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354303+0000", 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.277e-06 2024-09-15T00:09:19.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354310+0000", 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2450000000000003e-06 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354315+0000", 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0799999999999996e-06 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370290+0000", 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.015975552000000001 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370413+0000", 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012316399999999999 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370525+0000", 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000111537 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1355 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800139993, 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016460124999999999, 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1355 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354357+0000", 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354359+0000", 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0590000000000001e-06 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354365+0000", 2024-09-15T00:09:19.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7769999999999999e-06 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354370+0000", 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7849999999999999e-06 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370550+0000", 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016180486000000001 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370595+0000", 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5055999999999997e-05 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370817+0000", 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022162199999999999 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1356 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.254 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800121665000001, 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016655262000000001, 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1356 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354375+0000", 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5600000000000001e-07 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354377+0000", 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7710000000000001e-06 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354383+0000", 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8429999999999998e-06 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354388+0000", 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.583e-06 2024-09-15T00:09:19.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370848+0000", 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016460174000000001 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.370934+0000", 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6431e-05 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371030+0000", 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6003999999999998e-05 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1366 3.b 3:d2ce6848:::rbd_data.1129feb48f4b.000000000000000e:4 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.800031085000001, 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016710004000000001, 2024-09-15T00:09:19.259 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1366 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354466+0000", 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354468+0000", 2024-09-15T00:09:19.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.159e-06 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354474+0000", 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2720000000000003e-06 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354479+0000", 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8420000000000004e-06 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371054+0000", 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.016575185999999999 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371110+0000", 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5356999999999999e-05 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.371176+0000", 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5801000000000002e-05 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.263 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.264 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.264 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:19.265 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-15T00:09:19.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:19.265 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:19.263+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:19.266 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:19.267 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:19.268 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:19.366 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:19.365+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:19.451 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:19.451 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:19.451 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:19.451 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:19.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:19.467 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:19.465+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:19.568 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:19.566+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:19.609 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:19.609 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:19.610 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:19.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:19.668 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:19.667+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:19.743 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:19.743 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:19.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1260 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2023424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387845438999999, 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027065815, 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1260 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352960+0000", 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9559999999999997e-06 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352968+0000", 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5939999999999996e-06 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352973+0000", 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8409999999999999e-06 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379911+0000", 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.026938004000000002 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379958+0000", 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7607e-05 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380018+0000", 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9438999999999998e-05 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1269 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387812286000001, 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027165930000000001, 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1269 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9100000000000004e-07 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352988+0000", 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4779999999999998e-06 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353002+0000", 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3699999999999999e-05 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353007+0000", 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5310000000000001e-06 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380046+0000", 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027038801000000001 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380090+0000", 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:09:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380151+0000", 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1210000000000005e-05 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1277 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387781224999999, 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027261463999999999, 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1277 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353026+0000", 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3379999999999994e-06 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353034+0000", 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8790000000000003e-06 2024-09-15T00:09:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353039+0000", 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4489999999999998e-06 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380180+0000", 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.02714068 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380226+0000", 2024-09-15T00:09:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5776999999999999e-05 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380277+0000", 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1922000000000002e-05 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1285 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387740481, 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027368669000000002, 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1285 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1399999999999997e-07 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353060+0000", 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4310000000000001e-06 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353067+0000", 2024-09-15T00:09:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.746e-06 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353073+0000", 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.541e-06 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380305+0000", 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0272325 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380349+0000", 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3995000000000001e-05 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380425+0000", 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5941999999999999e-05 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1286 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387716566, 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027475013999999999, 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1286 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353087+0000", 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1299999999999998e-06 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353089+0000", 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.587e-06 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353096+0000", 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1409999999999999e-06 2024-09-15T00:09:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353102+0000", 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380455+0000", 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027352936000000001 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380501+0000", 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6236999999999998e-05 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380556+0000", 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4287999999999999e-05 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1292 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387683359, 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027570794999999999, 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1292 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4700000000000001e-07 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353117+0000", 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.295e-06 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353130+0000", 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2738999999999999e-05 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353141+0000", 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1885000000000001e-05 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380585+0000", 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027443263999999998 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380628+0000", 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3115999999999999e-05 2024-09-15T00:09:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380685+0000", 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6848999999999997e-05 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1294 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387647128999999, 2024-09-15T00:09:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027661577, 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1294 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353151+0000", 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353153+0000", 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353160+0000", 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353165+0000", 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0340000000000004e-06 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380714+0000", 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027549081 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380758+0000", 2024-09-15T00:09:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4323000000000002e-05 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380812+0000", 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3414000000000001e-05 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1314 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387391248, 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027532068, 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1314 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353407+0000", 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6300000000000004e-07 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353411+0000", 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0099999999999997e-06 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353421+0000", 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0309000000000001e-05 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353428+0000", 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7750000000000004e-06 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380839+0000", 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027410994000000001 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380883+0000", 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4079000000000003e-05 2024-09-15T00:09:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380938+0000", 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5138e-05 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1320 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387357914000001, 2024-09-15T00:09:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027624342999999999, 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1320 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353440+0000", 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0399999999999996e-07 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353442+0000", 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3800000000000001e-06 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353449+0000", 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3250000000000002e-06 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353475+0000", 2024-09-15T00:09:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.584e-05 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380968+0000", 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027492249999999999 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381010+0000", 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2228999999999998e-05 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381064+0000", 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3814999999999999e-05 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1321 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387311061, 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027730890000000001, 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1321 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353487+0000", 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6499999999999999e-07 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353489+0000", 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7240000000000001e-06 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353504+0000", 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4636e-05 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353510+0000", 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7060000000000001e-06 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381091+0000", 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027581668 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381158+0000", 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6229000000000006e-05 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381217+0000", 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9361999999999998e-05 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1324 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387042086999999, 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027587508, 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1324 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353756+0000", 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6800000000000009e-07 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353760+0000", 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1099999999999996e-06 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353772+0000", 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1464000000000001e-05 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353779+0000", 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3470000000000001e-06 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381246+0000", 2024-09-15T00:09:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027467155 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381287+0000", 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0553999999999998e-05 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381343+0000", 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5909999999999998e-05 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1332 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.387007007999999, 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027695646000000001, 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1332 2024-09-15T00:09:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353791+0000", 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.58e-07 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353793+0000", 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3350000000000001e-06 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353800+0000", 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2599999999999999e-06 2024-09-15T00:09:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353806+0000", 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6470000000000003e-06 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381370+0000", 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027564298000000001 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381429+0000", 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9302999999999998e-05 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381486+0000", 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6344999999999997e-05 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1333 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.386981708, 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027796923000000001, 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1333 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353816+0000", 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353818+0000", 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.137e-06 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353824+0000", 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3439999999999997e-06 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353829+0000", 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381516+0000", 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027686319000000001 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381558+0000", 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1927e-05 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381612+0000", 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4778999999999998e-05 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1353 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2039808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.386923456, 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027865687, 2024-09-15T00:09:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1353 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.99e-07 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353879+0000", 2024-09-15T00:09:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4089999999999996e-06 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353887+0000", 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1470000000000001e-06 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353893+0000", 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6840000000000001e-06 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381640+0000", 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027747720999999999 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381683+0000", 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2122000000000001e-05 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381739+0000", 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6904999999999998e-05 2024-09-15T00:09:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1359 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.38672087, 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.02778951, 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1359 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354077+0000", 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7000000000000003e-07 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354090+0000", 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3158e-05 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354102+0000", 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1572e-05 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354115+0000", 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3297e-05 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381769+0000", 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027653499000000002 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381811+0000", 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2348000000000002e-05 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381866+0000", 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4765999999999997e-05 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1360 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.386670515, 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027863138999999999, 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1360 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354130+0000", 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.542e-06 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354139+0000", 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.85e-06 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354144+0000", 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5550000000000002e-06 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381895+0000", 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027750654 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381937+0000", 2024-09-15T00:09:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2664999999999999e-05 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381990+0000", 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1363 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.386632295, 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027949321999999999, 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1363 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354166+0000", 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7199999999999998e-07 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354169+0000", 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.207e-06 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354178+0000", 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.738e-06 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354184+0000", 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1040000000000003e-06 2024-09-15T00:09:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382019+0000", 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027835069 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382059+0000", 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9988999999999999e-05 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382114+0000", 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5543e-05 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1370 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.386604793, 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028036106000000002, 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1370 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354193+0000", 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354195+0000", 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1090000000000001e-06 2024-09-15T00:09:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354210+0000", 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5423000000000001e-05 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354217+0000", 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2269999999999998e-06 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382138+0000", 2024-09-15T00:09:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027921069999999999 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382177+0000", 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9412999999999997e-05 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382228+0000", 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1437999999999998e-05 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1372 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.386568929999999, 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028122709999999999, 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1372 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354229+0000", 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8599999999999998e-07 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354232+0000", 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.757e-06 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354239+0000", 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3710000000000002e-06 2024-09-15T00:09:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354245+0000", 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6300000000000003e-06 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382259+0000", 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028014292999999999 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382301+0000", 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2354999999999997e-05 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382351+0000", 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9617999999999998e-05 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4574.0:298 3.8 3:13770e18:::rbd_data.112910301a74.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 7764992~33792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:14.691221+0000", 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.0495765539999997, 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.045417928000000003, 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4574", 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1245848753", 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 298 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.691221+0000", 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.691221+0000", 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.691222+0000", 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4079999999999999e-06 2024-09-15T00:09:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732525+0000", 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.041302740999999997 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732539+0000", 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3821e-05 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732544+0000", 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732595+0000", 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0726000000000002e-05 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732697+0000", 2024-09-15T00:09:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101612 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 1", 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.733016+0000", 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000319079 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736203+0000", 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031871040000000001 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736558+0000", 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00035541099999999998 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736593+0000", 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4715999999999998e-05 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736639+0000", 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5668999999999999e-05 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:19.844 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:19.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:19.846 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:19.843+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:19.855 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:09:19.910 INFO:teuthology.orchestra.run.smithi092.stdout:1141 read 0x2a78800 thru 0x2a862fd (0xdafe bytes) 2024-09-15T00:09:19.910 INFO:teuthology.orchestra.run.smithi092.stdout:1143 write 0x2e08000 thru 0x2e099d7 (0x19d8 bytes) 2024-09-15T00:09:19.911 INFO:teuthology.orchestra.run.smithi092.stdout:1144 read 0x4882000 thru 0x488713d (0x513e bytes) 2024-09-15T00:09:19.946 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:19.944+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:19.951 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:19.951 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:19.951 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:19.951 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:19.960 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:19.981 INFO:teuthology.orchestra.run.smithi092.stdout:1146 write 0xe25b800 thru 0xe25f476 (0x3c77 bytes) 2024-09-15T00:09:19.981 INFO:teuthology.orchestra.run.smithi092.stdout:1147 punch from 0x2092200 to 0x2092e00, (0xc00 bytes) 2024-09-15T00:09:19.981 INFO:teuthology.orchestra.run.smithi092.stdout:1148 read 0xc48d000 thru 0xc49b8b5 (0xe8b6 bytes) 2024-09-15T00:09:19.983 INFO:teuthology.orchestra.run.smithi092.stdout:1150 trunc from 0xe25f600 to 0xcd1c400 2024-09-15T00:09:20.018 INFO:teuthology.orchestra.run.smithi092.stdout:1152 read 0x8ff1200 thru 0x8ffbb82 (0xa983 bytes) 2024-09-15T00:09:20.018 INFO:teuthology.orchestra.run.smithi092.stdout:1155 read 0xa7eac00 thru 0xa7f4634 (0x9a35 bytes) 2024-09-15T00:09:20.023 INFO:teuthology.orchestra.run.smithi092.stdout:1160 trunc from 0xcd1c400 to 0x9205600 2024-09-15T00:09:20.046 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:20.045+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:20.056 INFO:teuthology.orchestra.run.smithi092.stdout:1162 read 0x6b17800 thru 0x6b22b24 (0xb325 bytes) 2024-09-15T00:09:20.056 INFO:teuthology.orchestra.run.smithi092.stdout:1163 write 0x7223800 thru 0x722565c (0x1e5d bytes) 2024-09-15T00:09:20.086 INFO:teuthology.orchestra.run.smithi092.stdout:1165 write 0xe47c200 thru 0xe47cd80 (0xb81 bytes) 2024-09-15T00:09:20.086 INFO:teuthology.orchestra.run.smithi092.stdout:1166 read 0xa753e00 thru 0xa75efc7 (0xb1c8 bytes) 2024-09-15T00:09:20.086 INFO:teuthology.orchestra.run.smithi092.stdout:1168 punch from 0x45a9400 to 0x45ac800, (0x3400 bytes) 2024-09-15T00:09:20.092 INFO:teuthology.orchestra.run.smithi092.stdout:1170 trunc from 0xe47ce00 to 0xa724c00 2024-09-15T00:09:20.121 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:20.122 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:20.122 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:20.122 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:20.122 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:20.123 INFO:teuthology.orchestra.run.smithi092.stdout:1173 read 0x3027a00 thru 0x303020d (0x880e bytes) 2024-09-15T00:09:20.125 INFO:teuthology.orchestra.run.smithi092.stdout:1174 read 0x8552800 thru 0x855284b (0x4c bytes) 2024-09-15T00:09:20.125 INFO:teuthology.orchestra.run.smithi092.stdout:1175 read 0x90eb600 thru 0x90f8a82 (0xd483 bytes) 2024-09-15T00:09:20.126 INFO:teuthology.orchestra.run.smithi092.stdout:1176 write 0x5c00 thru 0x10cd7 (0xb0d8 bytes) 2024-09-15T00:09:20.126 INFO:teuthology.orchestra.run.smithi092.stdout:1178 trunc from 0xa724c00 to 0xac08c00 2024-09-15T00:09:20.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:20.147 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:20.146+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:20.180 INFO:teuthology.orchestra.run.smithi092.stdout:1180 read 0x3a58e00 thru 0x3a64870 (0xba71 bytes) 2024-09-15T00:09:20.182 INFO:teuthology.orchestra.run.smithi092.stdout:1181 read 0x1927600 thru 0x192b6c1 (0x40c2 bytes) 2024-09-15T00:09:20.196 INFO:teuthology.orchestra.run.smithi092.stdout:1182 write 0xd58dc00 thru 0xd59c762 (0xeb63 bytes) 2024-09-15T00:09:20.201 INFO:teuthology.orchestra.run.smithi092.stdout:1183 trunc from 0xd59c800 to 0x9f32400 2024-09-15T00:09:20.249 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:20.247+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:20.252 INFO:teuthology.orchestra.run.smithi092.stdout:1184 write 0xeb0d800 thru 0xeb113cd (0x3bce bytes) 2024-09-15T00:09:20.258 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:20.258 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:20.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:20.259 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:20.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.259 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:976 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.259 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355630+0000", 2024-09-15T00:09:20.259 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900678813000001, 2024-09-15T00:09:20.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014900093999999999, 2024-09-15T00:09:20.259 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 976 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355630+0000", 2024-09-15T00:09:20.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355630+0000", 2024-09-15T00:09:20.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355631+0000", 2024-09-15T00:09:20.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:09:20.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355632+0000", 2024-09-15T00:09:20.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.716e-06 2024-09-15T00:09:20.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355639+0000", 2024-09-15T00:09:20.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3119999999999999e-06 2024-09-15T00:09:20.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355643+0000", 2024-09-15T00:09:20.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2400000000000001e-06 2024-09-15T00:09:20.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370458+0000", 2024-09-15T00:09:20.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014815439 2024-09-15T00:09:20.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370494+0000", 2024-09-15T00:09:20.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5988999999999997e-05 2024-09-15T00:09:20.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370530+0000", 2024-09-15T00:09:20.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6018e-05 2024-09-15T00:09:20.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.268 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:978 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 221184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.268 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355649+0000", 2024-09-15T00:09:20.268 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900660011999999, 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014981912999999999, 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 978 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355649+0000", 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355649+0000", 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355649+0000", 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355651+0000", 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.733e-06 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355657+0000", 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1489999999999999e-06 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355662+0000", 2024-09-15T00:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7489999999999998e-06 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370555+0000", 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014893356 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370591+0000", 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5722999999999997e-05 2024-09-15T00:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370631+0000", 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9807e-05 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:980 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.273 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355668+0000", 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900640962000001, 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015042632, 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 980 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355668+0000", 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355668+0000", 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355668+0000", 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0899999999999997e-07 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355670+0000", 2024-09-15T00:09:20.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6050000000000001e-06 2024-09-15T00:09:20.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355676+0000", 2024-09-15T00:09:20.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8309999999999997e-06 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355681+0000", 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1549999999999998e-06 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370643+0000", 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014962191999999999 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370676+0000", 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3042e-05 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370711+0000", 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4498000000000001e-05 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.279 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:982 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.279 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355687+0000", 2024-09-15T00:09:20.279 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900622035, 2024-09-15T00:09:20.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015125899, 2024-09-15T00:09:20.279 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.279 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.279 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.280 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 982 2024-09-15T00:09:20.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.280 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355687+0000", 2024-09-15T00:09:20.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355687+0000", 2024-09-15T00:09:20.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355687+0000", 2024-09-15T00:09:20.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-15T00:09:20.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355689+0000", 2024-09-15T00:09:20.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.731e-06 2024-09-15T00:09:20.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355695+0000", 2024-09-15T00:09:20.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1460000000000001e-06 2024-09-15T00:09:20.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355700+0000", 2024-09-15T00:09:20.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.758e-06 2024-09-15T00:09:20.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370735+0000", 2024-09-15T00:09:20.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015035389 2024-09-15T00:09:20.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370772+0000", 2024-09-15T00:09:20.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6480000000000003e-05 2024-09-15T00:09:20.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370813+0000", 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1022999999999997e-05 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:984 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355706+0000", 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900602972, 2024-09-15T00:09:20.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015314568000000001, 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 984 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355706+0000", 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355706+0000", 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355706+0000", 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355708+0000", 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7409999999999999e-06 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355714+0000", 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.037e-06 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355719+0000", 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.532e-06 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370858+0000", 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015139379 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.370943+0000", 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5043000000000006e-05 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371021+0000", 2024-09-15T00:09:20.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7438e-05 2024-09-15T00:09:20.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.292 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.292 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:986 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.292 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355953+0000", 2024-09-15T00:09:20.292 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.90035651, 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015173450999999999, 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 986 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355953+0000", 2024-09-15T00:09:20.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355953+0000", 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355953+0000", 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2300000000000001e-07 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355956+0000", 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0489999999999999e-06 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355963+0000", 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1980000000000004e-06 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355969+0000", 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7989999999999999e-06 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371050+0000", 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015080766000000001 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371085+0000", 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4885000000000002e-05 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371126+0000", 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1131000000000003e-05 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.296 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:988 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 303104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.355980+0000", 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900328718000001, 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015343035, 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 988 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355980+0000", 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355980+0000", 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355981+0000", 2024-09-15T00:09:20.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:09:20.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355983+0000", 2024-09-15T00:09:20.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-15T00:09:20.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355989+0000", 2024-09-15T00:09:20.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1569999999999997e-06 2024-09-15T00:09:20.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.355994+0000", 2024-09-15T00:09:20.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0610000000000003e-06 2024-09-15T00:09:20.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371169+0000", 2024-09-15T00:09:20.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015174488999999999 2024-09-15T00:09:20.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371247+0000", 2024-09-15T00:09:20.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8239999999999996e-05 2024-09-15T00:09:20.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371323+0000", 2024-09-15T00:09:20.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6588999999999995e-05 2024-09-15T00:09:20.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.304 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:991 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 319488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.304 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:20.305 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900308934, 2024-09-15T00:09:20.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015516135, 2024-09-15T00:09:20.305 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.306 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 991 2024-09-15T00:09:20.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.306 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:20.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356000+0000", 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5400000000000002e-07 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356008+0000", 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8679999999999999e-06 2024-09-15T00:09:20.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356015+0000", 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2809999999999997e-06 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356019+0000", 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8160000000000001e-06 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371364+0000", 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015344902000000001 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371439+0000", 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4666999999999995e-05 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371516+0000", 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7247000000000001e-05 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:993 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356039+0000", 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900270419, 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01567669, 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 993 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356039+0000", 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356039+0000", 2024-09-15T00:09:20.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356039+0000", 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8800000000000003e-07 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356041+0000", 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.17e-06 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356048+0000", 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7070000000000003e-06 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356053+0000", 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8949999999999997e-06 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371567+0000", 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015514457000000001 2024-09-15T00:09:20.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371643+0000", 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6148000000000003e-05 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371715+0000", 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1824999999999998e-05 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.315 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:995 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356059+0000", 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900249842000001, 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015775739, 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 995 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356059+0000", 2024-09-15T00:09:20.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356059+0000", 2024-09-15T00:09:20.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356060+0000", 2024-09-15T00:09:20.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-15T00:09:20.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356061+0000", 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.874e-06 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356067+0000", 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8200000000000002e-06 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356072+0000", 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5070000000000002e-06 2024-09-15T00:09:20.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371751+0000", 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015678877000000001 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371793+0000", 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1822e-05 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371835+0000", 2024-09-15T00:09:20.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2410999999999998e-05 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:996 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356077+0000", 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900232061000001, 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015873707000000001, 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 996 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356077+0000", 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356077+0000", 2024-09-15T00:09:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356077+0000", 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.39e-07 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356079+0000", 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7710000000000001e-06 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356085+0000", 2024-09-15T00:09:20.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7379999999999999e-06 2024-09-15T00:09:20.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356089+0000", 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5220000000000001e-06 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371850+0000", 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015761126 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371904+0000", 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3390999999999998e-05 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371951+0000", 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6820000000000002e-05 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:999 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356094+0000", 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900214564000001, 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015973820999999999, 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.326 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 999 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356094+0000", 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356094+0000", 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356095+0000", 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.27e-07 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356097+0000", 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356103+0000", 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0739999999999998e-06 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356108+0000", 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5889999999999996e-06 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.371977+0000", 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015869007000000001 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372018+0000", 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0525999999999998e-05 2024-09-15T00:09:20.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372068+0000", 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0735999999999997e-05 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1000 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900195946, 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016143586000000001, 2024-09-15T00:09:20.331 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1000 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356113+0000", 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1e-07 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356115+0000", 2024-09-15T00:09:20.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6759999999999999e-06 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356120+0000", 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2870000000000004e-06 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356125+0000", 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4750000000000004e-06 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372106+0000", 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015981278000000002 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372187+0000", 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0630999999999994e-05 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372257+0000", 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9929000000000001e-05 2024-09-15T00:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1003 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356131+0000", 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900177626, 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016310376000000001, 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1003 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356131+0000", 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356131+0000", 2024-09-15T00:09:20.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356132+0000", 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356134+0000", 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356140+0000", 2024-09-15T00:09:20.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4160000000000001e-06 2024-09-15T00:09:20.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356145+0000", 2024-09-15T00:09:20.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6059999999999996e-06 2024-09-15T00:09:20.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372289+0000", 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016144308999999999 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372335+0000", 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.562e-05 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372442+0000", 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106783 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1004 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356151+0000", 2024-09-15T00:09:20.341 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900157968, 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016474098999999999, 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1004 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356151+0000", 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356151+0000", 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356151+0000", 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356153+0000", 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9539999999999998e-06 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356159+0000", 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0809999999999999e-06 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356164+0000", 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4449999999999998e-06 2024-09-15T00:09:20.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372464+0000", 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016299783000000002 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372547+0000", 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3015000000000004e-05 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372625+0000", 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8471000000000001e-05 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1006 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900139208000001, 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016642545000000002, 2024-09-15T00:09:20.346 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1006 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356170+0000", 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356172+0000", 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-15T00:09:20.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356178+0000", 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0039999999999996e-06 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356183+0000", 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7509999999999999e-06 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372665+0000", 2024-09-15T00:09:20.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016482159 2024-09-15T00:09:20.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372740+0000", 2024-09-15T00:09:20.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4592999999999995e-05 2024-09-15T00:09:20.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372812+0000", 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2652999999999997e-05 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1008 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356188+0000", 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900120898000001, 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016831110999999999, 2024-09-15T00:09:20.351 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1008 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356188+0000", 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356188+0000", 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356188+0000", 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0499999999999999e-07 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356190+0000", 2024-09-15T00:09:20.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.838e-06 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356196+0000", 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8130000000000001e-06 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356201+0000", 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5170000000000001e-06 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372863+0000", 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016661974999999999 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.372943+0000", 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0055000000000005e-05 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373019+0000", 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6607999999999996e-05 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1011 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356206+0000", 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900103438, 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016992022999999998, 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.356 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1011 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356206+0000", 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356206+0000", 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356206+0000", 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2000000000000001e-07 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356208+0000", 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.733e-06 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356213+0000", 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6779999999999997e-06 2024-09-15T00:09:20.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356218+0000", 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4920000000000004e-06 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373057+0000", 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016839175000000001 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373128+0000", 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0994000000000005e-05 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373198+0000", 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9630999999999998e-05 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1014 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356223+0000", 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900086355999999, 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017112849999999999, 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1014 2024-09-15T00:09:20.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356223+0000", 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356223+0000", 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356223+0000", 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356225+0000", 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.759e-06 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356230+0000", 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6679999999999998e-06 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356235+0000", 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6489999999999999e-06 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373233+0000", 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016997486999999999 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373288+0000", 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5714999999999998e-05 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373336+0000", 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7240000000000002e-05 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4507.0:1015 3.14 3:2ccf3dfc:::rbd_data.1129aa143e51.000000000000002b:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:07.356250+0000", 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.900059355, 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017247518, 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4507", 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1015 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356250+0000", 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356250+0000", 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356250+0000", 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999998e-07 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356252+0000", 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1179999999999999e-06 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356258+0000", 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9580000000000004e-06 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.356263+0000", 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0919999999999997e-06 2024-09-15T00:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373361+0000", 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017098031 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373431+0000", 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9628000000000005e-05 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:07.373497+0000", 2024-09-15T00:09:20.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6112000000000007e-05 2024-09-15T00:09:20.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.370 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.370 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.370 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:20.374 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:20.372+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:20.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1185 read 0xcdf4e00 thru 0xcdf9c87 (0x4e88 bytes) 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1186 punch from 0x609dc00 to 0x60a4600, (0x6a00 bytes) 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1187 write 0x300ce00 thru 0x301a509 (0xd70a bytes) 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1188 read 0x90c2000 thru 0x90ce905 (0xc906 bytes) 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1189 trunc from 0xeb11400 to 0x9126200 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1191 write 0x7aca600 thru 0x7ad48a0 (0xa2a1 bytes) 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1192 write 0x508a400 thru 0x5097889 (0xd48a bytes) 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1193 read 0x37a6e00 thru 0x37b3f4e (0xd14f bytes) 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1194 read 0x1fb0e00 thru 0x1fbeeda (0xe0db bytes) 2024-09-15T00:09:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:1195 trunc from 0x9126200 to 0xc88a000 2024-09-15T00:09:20.394 INFO:teuthology.orchestra.run.smithi092.stdout:1198 read 0x43c2000 thru 0x43c84ed (0x64ee bytes) 2024-09-15T00:09:20.396 INFO:teuthology.orchestra.run.smithi092.stdout:1199 write 0x9ca4a00 thru 0x9ca936b (0x496c bytes) 2024-09-15T00:09:20.397 INFO:teuthology.orchestra.run.smithi092.stdout:1201 read 0x5ff600 thru 0x6044a7 (0x4ea8 bytes) 2024-09-15T00:09:20.399 INFO:teuthology.orchestra.run.smithi092.stdout:1202 trunc from 0xc88a000 to 0xdded800 2024-09-15T00:09:20.415 INFO:teuthology.orchestra.run.smithi092.stdout:1203 read 0x1cf0800 thru 0x1cfb1f6 (0xa9f7 bytes) 2024-09-15T00:09:20.431 INFO:teuthology.orchestra.run.smithi092.stdout:1205 trunc from 0xdded800 to 0x627a00 2024-09-15T00:09:20.474 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:20.472+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:20.519 INFO:teuthology.orchestra.run.smithi092.stdout:1208 trunc from 0x627a00 to 0xe380e00 2024-09-15T00:09:20.537 INFO:teuthology.orchestra.run.smithi092.stdout:1212 trunc from 0xe380e00 to 0x6ddaa00 2024-09-15T00:09:20.543 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:20.543 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:20.543 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:20.543 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:20.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:20.559 INFO:teuthology.orchestra.run.smithi092.stdout:1213 write 0x72e7600 thru 0x72f0a1e (0x941f bytes) 2024-09-15T00:09:20.590 INFO:teuthology.orchestra.run.smithi092.stdout:1215 write 0xde15000 thru 0xde1ab73 (0x5b74 bytes) 2024-09-15T00:09:20.591 INFO:teuthology.orchestra.run.smithi092.stdout:1219 write 0xc7f1400 thru 0xc7fd3ba (0xbfbb bytes) 2024-09-15T00:09:20.591 INFO:teuthology.orchestra.run.smithi092.stdout:1220 punch from 0x5edd400 to 0x5eddc00, (0x800 bytes) 2024-09-15T00:09:20.593 INFO:teuthology.orchestra.run.smithi092.stdout:1222 punch from 0xa169200 to 0xa178800, (0xf600 bytes) 2024-09-15T00:09:20.593 INFO:teuthology.orchestra.run.smithi092.stdout:1226 read 0x7bd6e00 thru 0x7bda707 (0x3908 bytes) 2024-09-15T00:09:20.593 INFO:teuthology.orchestra.run.smithi092.stdout:1228 write 0x1379800 thru 0x137a9cb (0x11cc bytes) 2024-09-15T00:09:20.593 INFO:teuthology.orchestra.run.smithi092.stdout:1229 write 0xd98c400 thru 0xd991020 (0x4c21 bytes) 2024-09-15T00:09:20.594 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:20.592+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:20.598 INFO:teuthology.orchestra.run.smithi092.stdout:1230 trunc from 0xde1ac00 to 0x9535600 2024-09-15T00:09:20.663 INFO:teuthology.orchestra.run.smithi092.stdout:1232 read 0x91a1400 thru 0x91aac22 (0x9823 bytes) 2024-09-15T00:09:20.676 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:20.675+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:20.683 INFO:teuthology.orchestra.run.smithi092.stdout:1234 write 0xecd3000 thru 0xecd9ca3 (0x6ca4 bytes) 2024-09-15T00:09:20.683 INFO:teuthology.orchestra.run.smithi092.stdout:1236 punch from 0xa5ac800 to 0xa5b3000, (0x6800 bytes) 2024-09-15T00:09:20.684 INFO:teuthology.orchestra.run.smithi092.stdout:1239 write 0xc390c00 thru 0xc396a78 (0x5e79 bytes) 2024-09-15T00:09:20.684 INFO:teuthology.orchestra.run.smithi092.stdout:1241 read 0xaca200 thru 0xad4d84 (0xab85 bytes) 2024-09-15T00:09:20.684 INFO:teuthology.orchestra.run.smithi092.stdout:1243 read 0xd8a3800 thru 0xd8a9af1 (0x62f2 bytes) 2024-09-15T00:09:20.685 INFO:teuthology.orchestra.run.smithi092.stdout:1244 write 0xcc5bc00 thru 0xcc6b42d (0xf82e bytes) 2024-09-15T00:09:20.685 INFO:teuthology.orchestra.run.smithi092.stdout:1246 read 0xa2fc600 thru 0xa2fdb57 (0x1558 bytes) 2024-09-15T00:09:20.685 INFO:teuthology.orchestra.run.smithi092.stdout:1247 read 0x59adc00 thru 0x59bcc33 (0xf034 bytes) 2024-09-15T00:09:20.685 INFO:teuthology.orchestra.run.smithi092.stdout:1249 read 0x27b8a00 thru 0x27bbdcc (0x33cd bytes) 2024-09-15T00:09:20.685 INFO:teuthology.orchestra.run.smithi092.stdout:1252 write 0xa62be00 thru 0xa6332b6 (0x74b7 bytes) 2024-09-15T00:09:20.685 INFO:teuthology.orchestra.run.smithi092.stdout:1253 write 0x5d11400 thru 0x5d1ec40 (0xd841 bytes) 2024-09-15T00:09:20.698 INFO:teuthology.orchestra.run.smithi092.stdout:1254 trunc from 0xecd9e00 to 0x3029000 2024-09-15T00:09:20.717 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:20.717 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:20.717 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:20.717 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:20.717 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:20.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:20.776 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:20.775+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:20.783 INFO:teuthology.orchestra.run.smithi092.stdout:1257 read 0x1b8c00 thru 0x1ba2d3 (0x16d4 bytes) 2024-09-15T00:09:20.784 INFO:teuthology.orchestra.run.smithi092.stdout:1261 read 0x2ef6c00 thru 0x2ef8369 (0x176a bytes) 2024-09-15T00:09:20.796 INFO:teuthology.orchestra.run.smithi092.stdout:1262 trunc from 0x3029000 to 0xd559c00 2024-09-15T00:09:20.806 INFO:teuthology.orchestra.run.smithi092.stdout:1263 write 0x26eb200 thru 0x26f0c14 (0x5a15 bytes) 2024-09-15T00:09:20.807 INFO:teuthology.orchestra.run.smithi092.stdout:1264 write 0xd214c00 thru 0xd22091a (0xbd1b bytes) 2024-09-15T00:09:20.807 INFO:teuthology.orchestra.run.smithi092.stdout:1265 read 0x7536000 thru 0x753ec39 (0x8c3a bytes) 2024-09-15T00:09:20.809 INFO:teuthology.orchestra.run.smithi092.stdout:1267 trunc from 0xd559c00 to 0xbce4200 2024-09-15T00:09:20.846 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:20.846 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3427 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55668362900000001, 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027724631, 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3427 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:20.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287582+0000", 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2599999999999998e-07 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287584+0000", 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8759999999999999e-06 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287589+0000", 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287594+0000", 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.357e-06 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315217+0000", 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027622785 2024-09-15T00:09:20.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315257+0000", 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0228000000000002e-05 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315306+0000", 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9228999999999999e-05 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3428 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55666562399999997, 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027928562000000001, 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3428 2024-09-15T00:09:20.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:20.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287600+0000", 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287602+0000", 2024-09-15T00:09:20.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287608+0000", 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287613+0000", 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8740000000000003e-06 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315346+0000", 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027733099000000001 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315431+0000", 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4350000000000004e-05 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315528+0000", 2024-09-15T00:09:20.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7374000000000007e-05 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3429 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55664631399999998, 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028028630999999998, 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3429 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287621+0000", 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287627+0000", 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8440000000000003e-06 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287631+0000", 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5009999999999998e-06 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315556+0000", 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027925241999999999 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315595+0000", 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8512e-05 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315647+0000", 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2318000000000003e-05 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3430 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:20.862 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55662824, 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028122114, 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3430 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287644+0000", 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4219999999999997e-06 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287650+0000", 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0920000000000003e-06 2024-09-15T00:09:20.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287654+0000", 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4279999999999998e-06 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315671+0000", 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016592 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315709+0000", 2024-09-15T00:09:20.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8231e-05 2024-09-15T00:09:20.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315759+0000", 2024-09-15T00:09:20.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9926999999999998e-05 2024-09-15T00:09:20.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3431 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55660469999999995, 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02831842, 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3431 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287661+0000", 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287663+0000", 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9570000000000001e-06 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287668+0000", 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287673+0000", 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315799+0000", 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028125741999999999 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315891+0000", 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2701000000000006e-05 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315979+0000", 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7287000000000006e-05 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3432 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55658631599999997, 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028430624000000002, 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3432 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.873 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287681+0000", 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8530000000000001e-06 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287686+0000", 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4659999999999998e-06 2024-09-15T00:09:20.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287691+0000", 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2230000000000001e-06 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316006+0000", 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028315344999999999 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316054+0000", 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8004999999999999e-05 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316109+0000", 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5374999999999998e-05 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3433 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.877 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55656855999999999, 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028528593000000001, 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3433 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287697+0000", 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287699+0000", 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8509999999999999e-06 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287704+0000", 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287710+0000", 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316124+0000", 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028414466999999999 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316171+0000", 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6783999999999998e-05 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316225+0000", 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3891999999999998e-05 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3434 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55654917500000001, 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028632801999999999, 2024-09-15T00:09:20.882 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3434 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-07 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287718+0000", 2024-09-15T00:09:20.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287724+0000", 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8390000000000003e-06 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287729+0000", 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8389999999999997e-06 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316250+0000", 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028521135 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316295+0000", 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4540999999999999e-05 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316349+0000", 2024-09-15T00:09:20.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4083999999999999e-05 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3435 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55651849900000006, 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028723243999999998, 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3435 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287747+0000", 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4900000000000002e-07 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287748+0000", 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4190000000000001e-06 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287752+0000", 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7900000000000001e-06 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287756+0000", 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1609999999999996e-06 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316375+0000", 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028618932 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316417+0000", 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316470+0000", 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3436 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55650345099999998, 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028827854, 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3436 2024-09-15T00:09:20.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287766+0000", 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287771+0000", 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5140000000000001e-06 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287776+0000", 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316494+0000", 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028718144000000001 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316538+0000", 2024-09-15T00:09:20.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4129999999999999e-05 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316589+0000", 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1548999999999997e-05 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3437 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55648555899999996, 2024-09-15T00:09:20.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028972188999999999, 2024-09-15T00:09:20.897 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.897 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.897 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.897 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3437 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287780+0000", 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287782+0000", 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8419999999999999e-06 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287787+0000", 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3059999999999997e-06 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287791+0000", 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9920000000000001e-06 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316615+0000", 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028824487999999999 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316668+0000", 2024-09-15T00:09:20.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2883999999999997e-05 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316752+0000", 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3308999999999998e-05 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3438 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.901 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.556468517, 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029124331999999999, 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3438 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287797+0000", 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287799+0000", 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287804+0000", 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7069999999999997e-06 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287808+0000", 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0720000000000001e-06 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316799+0000", 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028990241999999999 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316845+0000", 2024-09-15T00:09:20.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5837000000000002e-05 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316921+0000", 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6255999999999996e-05 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3439 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:20.906 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55645097799999999, 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029290665, 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3439 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:20.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287816+0000", 2024-09-15T00:09:20.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6980000000000001e-06 2024-09-15T00:09:20.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287824+0000", 2024-09-15T00:09:20.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2500000000000006e-06 2024-09-15T00:09:20.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287829+0000", 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316967+0000", 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029138319999999999 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317047+0000", 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0081999999999997e-05 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317105+0000", 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7497999999999997e-05 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.911 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3440 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55642991900000005, 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029395451999999999, 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3440 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287837+0000", 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287843+0000", 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6799999999999998e-06 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287848+0000", 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317131+0000", 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029283337999999999 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317176+0000", 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5176999999999998e-05 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317231+0000", 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4120999999999999e-05 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3441 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55641185500000001, 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029496321999999998, 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3441 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:09:20.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287855+0000", 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7099999999999999e-06 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287861+0000", 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287866+0000", 2024-09-15T00:09:20.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317252+0000", 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029386051999999999 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317297+0000", 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5164999999999999e-05 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317349+0000", 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2534000000000002e-05 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3442 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55639258899999999, 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029594697999999999, 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3442 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287873+0000", 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287879+0000", 2024-09-15T00:09:20.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287885+0000", 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.755e-06 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287890+0000", 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317375+0000", 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029485457 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317416+0000", 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1217999999999997e-05 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317467+0000", 2024-09-15T00:09:20.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0819999999999998e-05 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3443 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55636925000000004, 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029684676, 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3443 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287898+0000", 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:09:20.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287904+0000", 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8860000000000001e-06 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287908+0000", 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5650000000000003e-06 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317488+0000", 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029579437 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317530+0000", 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1900999999999997e-05 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317580+0000", 2024-09-15T00:09:20.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0603000000000003e-05 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3444 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55634403200000004, 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029772823, 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.931 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3444 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287931+0000", 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0008e-05 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287938+0000", 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287943+0000", 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317603+0000", 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029659913999999999 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317646+0000", 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2367999999999998e-05 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317694+0000", 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8201999999999997e-05 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3445 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.556314694, 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029863565000000002, 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3445 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287951+0000", 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287953+0000", 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.765e-06 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287958+0000", 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999996e-06 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287963+0000", 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.634e-06 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317714+0000", 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029751353000000001 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317755+0000", 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0342000000000001e-05 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317814+0000", 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9188999999999999e-05 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.940 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3446 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.55629639600000003, 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029967884, 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3446 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:20.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:09:20.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287971+0000", 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287977+0000", 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4330000000000003e-06 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287981+0000", 2024-09-15T00:09:20.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3989999999999997e-06 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317837+0000", 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029855942 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317878+0000", 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467000000000001e-05 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:20.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:20.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:20.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317936+0000", 2024-09-15T00:09:20.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8100999999999998e-05 2024-09-15T00:09:20.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.945 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:20.945 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:20.945 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:20.948 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:09:20.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:20.948 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:20.945+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:20.949 INFO:teuthology.orchestra.run.smithi092.stdout:1269 trunc from 0xbce4200 to 0xe3f4c00 2024-09-15T00:09:20.949 INFO:teuthology.orchestra.run.smithi092.stdout:1270 read 0x448c000 thru 0x448cbae (0xbaf bytes) 2024-09-15T00:09:20.949 INFO:teuthology.orchestra.run.smithi092.stdout:1271 read 0x59a4600 thru 0x59a4a7a (0x47b bytes) 2024-09-15T00:09:20.949 INFO:teuthology.orchestra.run.smithi092.stdout:1272 trunc from 0xe3f4c00 to 0xac2000 2024-09-15T00:09:20.949 INFO:teuthology.orchestra.run.smithi092.stdout:1273 read 0x955e00 thru 0x963af5 (0xdcf6 bytes) 2024-09-15T00:09:20.949 INFO:teuthology.orchestra.run.smithi092.stdout:1274 punch from 0x4a4200 to 0x4a8e00, (0x4c00 bytes) 2024-09-15T00:09:20.949 INFO:teuthology.orchestra.run.smithi092.stdout:1275 read 0xa27200 thru 0xa2adc9 (0x3bca bytes) 2024-09-15T00:09:20.949 INFO:teuthology.orchestra.run.smithi092.stdout:1279 write 0x8cdce00 thru 0x8ce9ede (0xd0df bytes) 2024-09-15T00:09:20.950 INFO:teuthology.orchestra.run.smithi092.stdout:1281 write 0x7a7f800 thru 0x7a830d5 (0x38d6 bytes) 2024-09-15T00:09:20.950 INFO:teuthology.orchestra.run.smithi092.stdout:1282 trunc from 0x8cea000 to 0x905c000 2024-09-15T00:09:20.972 INFO:teuthology.orchestra.run.smithi092.stdout:1284 write 0x143fc00 thru 0x1445240 (0x5641 bytes) 2024-09-15T00:09:20.980 INFO:teuthology.orchestra.run.smithi092.stdout:1285 punch from 0x2189800 to 0x2191c00, (0x8400 bytes) 2024-09-15T00:09:20.982 INFO:teuthology.orchestra.run.smithi092.stdout:1286 read 0x78c6800 thru 0x78ca208 (0x3a09 bytes) 2024-09-15T00:09:20.989 INFO:teuthology.orchestra.run.smithi092.stdout:1288 trunc from 0x905c000 to 0x3231e00 2024-09-15T00:09:21.022 INFO:teuthology.orchestra.run.smithi092.stdout:1291 write 0x1a5b600 thru 0x1a6a27c (0xec7d bytes) 2024-09-15T00:09:21.030 INFO:teuthology.orchestra.run.smithi092.stdout:1294 read 0x5da00 thru 0x6bcf3 (0xe2f4 bytes) 2024-09-15T00:09:21.034 INFO:teuthology.orchestra.run.smithi092.stdout:1295 trunc from 0x3231e00 to 0x1031c00 2024-09-15T00:09:21.047 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:21.046+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:21.072 INFO:teuthology.orchestra.run.smithi092.stdout:1296 write 0x887c00 thru 0x88acde (0x30df bytes) 2024-09-15T00:09:21.077 INFO:teuthology.orchestra.run.smithi092.stdout:1297 read 0xfe0600 thru 0xfe58fc (0x52fd bytes) 2024-09-15T00:09:21.077 INFO:teuthology.orchestra.run.smithi092.stdout:1300 write 0x2dac00 thru 0x2de107 (0x3508 bytes) 2024-09-15T00:09:21.092 INFO:teuthology.orchestra.run.smithi092.stdout:1301 write 0x3d6a600 thru 0x3d77352 (0xcd53 bytes) 2024-09-15T00:09:21.098 INFO:teuthology.orchestra.run.smithi092.stdout:1306 trunc from 0x3d77400 to 0x8b14200 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4622.0:8571 3.2 e38/20 3:424e8a73:::rbd_data.112981949474.00000000000000f1:head v 38'141, mlcod=38'139)", 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:21.104072+0000", 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00084672100000000004, 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086696199999999998, 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.106 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:21.104072+0000", 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:21.104072+0000", 2024-09-15T00:09:21.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:21.104072+0000", 2024-09-15T00:09:21.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1699999999999997e-07 2024-09-15T00:09:21.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:21.104082+0000", 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5219999999999996e-06 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:21.104095+0000", 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3380000000000001e-05 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:21.104102+0000", 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7920000000000004e-06 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:21.104145+0000", 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3248999999999999e-05 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:21.104158+0000", 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2904e-05 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:09:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-15T00:09:21.111 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:21.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:21.121 INFO:teuthology.orchestra.run.smithi092.stdout:1308 read 0x207d000 thru 0x208132f (0x4330 bytes) 2024-09-15T00:09:21.139 INFO:teuthology.orchestra.run.smithi092.stdout:1311 write 0xd470600 thru 0xd47b046 (0xaa47 bytes) 2024-09-15T00:09:21.140 INFO:teuthology.orchestra.run.smithi092.stdout:1313 write 0x7d87e00 thru 0x7d8b8d7 (0x3ad8 bytes) 2024-09-15T00:09:21.140 INFO:teuthology.orchestra.run.smithi092.stdout:1314 read 0x4aa1800 thru 0x4aa94fe (0x7cff bytes) 2024-09-15T00:09:21.140 INFO:teuthology.orchestra.run.smithi092.stdout:1315 write 0x745e400 thru 0x746ae64 (0xca65 bytes) 2024-09-15T00:09:21.140 INFO:teuthology.orchestra.run.smithi092.stdout:1317 write 0x10e7000 thru 0x10f0907 (0x9908 bytes) 2024-09-15T00:09:21.140 INFO:teuthology.orchestra.run.smithi092.stdout:1322 write 0x8d18a00 thru 0x8d25795 (0xcd96 bytes) 2024-09-15T00:09:21.140 INFO:teuthology.orchestra.run.smithi092.stdout:1323 read 0x967c400 thru 0x9686ac6 (0xa6c7 bytes) 2024-09-15T00:09:21.141 INFO:teuthology.orchestra.run.smithi092.stdout:1325 write 0xd1c0000 thru 0xd1c1ca9 (0x1caa bytes) 2024-09-15T00:09:21.141 INFO:teuthology.orchestra.run.smithi092.stdout:1326 read 0xa670000 thru 0xa67196e (0x196f bytes) 2024-09-15T00:09:21.141 INFO:teuthology.orchestra.run.smithi092.stdout:1327 write 0x9a5ea00 thru 0x9a6043b (0x1a3c bytes) 2024-09-15T00:09:21.141 INFO:teuthology.orchestra.run.smithi092.stdout:1328 write 0x178da00 thru 0x1795a7b (0x807c bytes) 2024-09-15T00:09:21.152 INFO:teuthology.orchestra.run.smithi092.stdout:1331 trunc from 0xd47b200 to 0x38dae00 2024-09-15T00:09:21.160 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:21.158+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:21.239 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:21.239 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:21.239 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:21.239 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:21.239 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:21.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:21.251 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:21.250+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:21.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:21.278 INFO:teuthology.orchestra.run.smithi092.stdout:1333 write 0x7449a00 thru 0x7454668 (0xac69 bytes) 2024-09-15T00:09:21.278 INFO:teuthology.orchestra.run.smithi092.stdout:1334 punch from 0x66c1c00 to 0x66d1200, (0xf600 bytes) 2024-09-15T00:09:21.285 INFO:teuthology.orchestra.run.smithi092.stdout:1336 trunc from 0x7454800 to 0xbfbee00 2024-09-15T00:09:21.321 INFO:teuthology.orchestra.run.smithi092.stdout:1337 read 0x743f400 thru 0x743f963 (0x564 bytes) 2024-09-15T00:09:21.322 INFO:teuthology.orchestra.run.smithi092.stdout:1339 write 0x9e35e00 thru 0x9e40bf6 (0xadf7 bytes) 2024-09-15T00:09:21.352 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:21.351+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:21.353 INFO:teuthology.orchestra.run.smithi092.stdout:1341 write 0xd8dfe00 thru 0xd8ed673 (0xd874 bytes) 2024-09-15T00:09:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1343 write 0x2ccb400 thru 0x2ccbe34 (0xa35 bytes) 2024-09-15T00:09:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1345 read 0x9828000 thru 0x982f8ad (0x78ae bytes) 2024-09-15T00:09:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1346 write 0x2abac00 thru 0x2ac2324 (0x7725 bytes) 2024-09-15T00:09:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1347 read 0x4a7c600 thru 0x4a7ed29 (0x272a bytes) 2024-09-15T00:09:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1349 write 0xa62d400 thru 0xa63c37a (0xef7b bytes) 2024-09-15T00:09:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1351 read 0x55d2400 thru 0x55d8104 (0x5d05 bytes) 2024-09-15T00:09:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1352 read 0x22a4c00 thru 0x22a93b4 (0x47b5 bytes) 2024-09-15T00:09:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1353 read 0xb736e00 thru 0xb738fcc (0x21cd bytes) 2024-09-15T00:09:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:1354 read 0x3d7e00 thru 0x3de6b8 (0x68b9 bytes) 2024-09-15T00:09:21.356 INFO:teuthology.orchestra.run.smithi092.stdout:1356 read 0x1546000 thru 0x1555378 (0xf379 bytes) 2024-09-15T00:09:21.357 INFO:teuthology.orchestra.run.smithi092.stdout:1357 write 0x5d3000 thru 0x5deb7f (0xbb80 bytes) 2024-09-15T00:09:21.357 INFO:teuthology.orchestra.run.smithi092.stdout:1360 read 0xb14ce00 thru 0xb1500e0 (0x32e1 bytes) 2024-09-15T00:09:21.357 INFO:teuthology.orchestra.run.smithi092.stdout:1361 punch from 0xaed8a00 to 0xaed8e00, (0x400 bytes) 2024-09-15T00:09:21.363 INFO:teuthology.orchestra.run.smithi092.stdout:1363 punch from 0x4596200 to 0x4598800, (0x2600 bytes) 2024-09-15T00:09:21.363 INFO:teuthology.orchestra.run.smithi092.stdout:1365 read 0xb612c00 thru 0xb61dbf2 (0xaff3 bytes) 2024-09-15T00:09:21.364 INFO:teuthology.orchestra.run.smithi092.stdout:1367 punch from 0xa9b6200 to 0xa9bca00, (0x6800 bytes) 2024-09-15T00:09:21.364 INFO:teuthology.orchestra.run.smithi092.stdout:1369 punch from 0x4bdbc00 to 0x4be4e00, (0x9200 bytes) 2024-09-15T00:09:21.365 INFO:teuthology.orchestra.run.smithi092.stdout:1371 flatten 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1142 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5389719309999998, 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045190363999999997, 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1142 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882756+0000", 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882773+0000", 2024-09-15T00:09:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6872999999999999e-05 2024-09-15T00:09:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882780+0000", 2024-09-15T00:09:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4240000000000004e-06 2024-09-15T00:09:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882787+0000", 2024-09-15T00:09:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8510000000000001e-06 2024-09-15T00:09:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927799+0000", 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045011974000000003 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927843+0000", 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4270999999999997e-05 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927946+0000", 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102581 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5389232389999998, 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:09:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5388846090000001, 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5388586719999999, 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:09:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5388426040000001, 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:09:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:09:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.538701509, 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:09:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:09:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5379297540000003, 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:09:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:09:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:09:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:09:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:09:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5378447230000001, 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5378097070000001, 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:09:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:09:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:09:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:09:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:09:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.537784104, 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:09:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:09:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5376098379999998, 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:09:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:09:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:09:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:09:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5373786989999996, 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:09:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1157 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5365972609999998, 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045184281, 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1157 2024-09-15T00:09:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885132+0000", 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885139+0000", 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0350000000000004e-06 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885153+0000", 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3515e-05 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930177+0000", 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045024495999999997 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930249+0000", 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2072999999999999e-05 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930314+0000", 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4826999999999994e-05 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.536566755, 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:09:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5364870149999996, 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:09:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:09:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:09:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:09:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:09:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.53628199, 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:09:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:09:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5360704570000001, 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:09:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:09:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:09:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:09:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:09:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5335144500000002, 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:09:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:09:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5315313440000002, 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:09:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:09:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:09:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5315117730000001, 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:09:21.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:21.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:09:21.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:09:21.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:21.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:21.529 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:21.531 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:21.532 INFO:teuthology.orchestra.run.smithi092.stdout:1372 punch from 0xb0fcc00 to 0xb107600, (0xaa00 bytes) 2024-09-15T00:09:21.532 INFO:teuthology.orchestra.run.smithi092.stdout:1373 punch from 0x590400 to 0x59a600, (0xa200 bytes) 2024-09-15T00:09:21.532 INFO:teuthology.orchestra.run.smithi092.stdout:1375 write 0x5b2f600 thru 0x5b3d3d1 (0xddd2 bytes) 2024-09-15T00:09:21.532 INFO:teuthology.orchestra.run.smithi092.stdout:1376 trunc from 0xd8ed800 to 0x6ca5400 2024-09-15T00:09:21.532 INFO:teuthology.orchestra.run.smithi092.stdout:1379 read 0x6c3e00 thru 0x6d28fc (0xeafd bytes) 2024-09-15T00:09:21.532 INFO:teuthology.orchestra.run.smithi092.stdout:1381 clone 11 order 21 su 131072 sc 4 2024-09-15T00:09:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:21.533 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:21.528+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:21.533 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:21.533 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:21.534 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:21.534 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:21.535 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:21.541 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:21.541 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:21.631 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:21.629+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:21.642 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:21.642 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:21.651 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:21.732 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:21.731+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:21.758 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:21.758 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:21.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:21.833 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:21.831+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:21.920 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:21.920 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:21.930 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:21.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:21.973+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:22.035 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:22.033+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:22.038 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:22.039 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:22.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:22.135 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:22.133+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:22.199 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:22.199 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:22.209 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:22.236 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:22.235+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:22.237 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-15T00:09:22.306 INFO:teuthology.orchestra.run.smithi092.stdout:1382 read 0x4430200 thru 0x4431bc3 (0x19c4 bytes) 2024-09-15T00:09:22.313 INFO:teuthology.orchestra.run.smithi092.stdout:1384 trunc from 0x6ca5400 to 0xc3bd400 2024-09-15T00:09:22.318 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:22.318 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:22.326 INFO:teuthology.orchestra.run.smithi092.stdout:1386 write 0x4077e00 thru 0x4079353 (0x1554 bytes) 2024-09-15T00:09:22.326 INFO:teuthology.orchestra.run.smithi092.stdout:1387 write 0x424aa00 thru 0x425689c (0xbe9d bytes) 2024-09-15T00:09:22.328 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:22.333 INFO:teuthology.orchestra.run.smithi092.stdout:1388 trunc from 0xc3bd400 to 0x65f8400 2024-09-15T00:09:22.373 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:22.372+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:22.373 INFO:teuthology.orchestra.run.smithi092.stdout:1390 trunc from 0x65f8400 to 0x9284400 2024-09-15T00:09:22.383 INFO:teuthology.orchestra.run.smithi092.stdout:1393 read 0x5516600 thru 0x5522c89 (0xc68a bytes) 2024-09-15T00:09:22.397 INFO:teuthology.orchestra.run.smithi092.stdout:1395 write 0xc5e9400 thru 0xc5ed06e (0x3c6f bytes) 2024-09-15T00:09:22.422 INFO:teuthology.orchestra.run.smithi092.stdout:1396 write 0xe2e5800 thru 0xe2ec046 (0x6847 bytes) 2024-09-15T00:09:22.429 INFO:teuthology.orchestra.run.smithi092.stdout:1397 write 0x7554800 thru 0x7558d8f (0x4590 bytes) 2024-09-15T00:09:22.429 INFO:teuthology.orchestra.run.smithi092.stdout:1398 write 0xcbc7400 thru 0xcbd5e12 (0xea13 bytes) 2024-09-15T00:09:22.429 INFO:teuthology.orchestra.run.smithi092.stdout:1399 write 0xbc01000 thru 0xbc052b3 (0x42b4 bytes) 2024-09-15T00:09:22.429 INFO:teuthology.orchestra.run.smithi092.stdout:1401 read 0x941a600 thru 0x941fafc (0x54fd bytes) 2024-09-15T00:09:22.429 INFO:teuthology.orchestra.run.smithi092.stdout:1402 read 0xdd9800 thru 0xde77df (0xdfe0 bytes) 2024-09-15T00:09:22.429 INFO:teuthology.orchestra.run.smithi092.stdout:1403 punch from 0x4b71400 to 0x4b73400, (0x2000 bytes) 2024-09-15T00:09:22.438 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:22.437+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:22.442 INFO:teuthology.orchestra.run.smithi092.stdout:1404 trunc from 0xe2ec200 to 0x3bb7600 2024-09-15T00:09:22.459 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:22.460 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:22.469 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:22.485 INFO:teuthology.orchestra.run.smithi092.stdout:1407 write 0xc78da00 thru 0xc792dea (0x53eb bytes) 2024-09-15T00:09:22.495 INFO:teuthology.orchestra.run.smithi092.stdout:1408 trunc from 0xc792e00 to 0x49cc600 2024-09-15T00:09:22.513 INFO:teuthology.orchestra.run.smithi092.stdout:1409 trunc from 0x49cc600 to 0x267c200 2024-09-15T00:09:22.534 INFO:teuthology.orchestra.run.smithi092.stdout:1410 punch from 0x29200 to 0x2c400, (0x3200 bytes) 2024-09-15T00:09:22.537 INFO:teuthology.orchestra.run.smithi092.stdout:1411 read 0x12e0a00 thru 0x12e0ff7 (0x5f8 bytes) 2024-09-15T00:09:22.547 INFO:teuthology.orchestra.run.smithi092.stdout:1412 trunc from 0x267c200 to 0xae12000 2024-09-15T00:09:22.554 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:22.553+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:22.571 INFO:teuthology.orchestra.run.smithi092.stdout:1414 trunc from 0xae12000 to 0x3a3c00 2024-09-15T00:09:22.618 INFO:teuthology.orchestra.run.smithi092.stdout:1417 read 0x12e200 thru 0x13d788 (0xf589 bytes) 2024-09-15T00:09:22.618 INFO:teuthology.orchestra.run.smithi092.stdout:1418 read 0x197400 thru 0x1a265e (0xb25f bytes) 2024-09-15T00:09:22.640 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:22.641 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:22.641 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:22.639+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:22.646 INFO:teuthology.orchestra.run.smithi092.stdout:1419 write 0xab0d400 thru 0xab11006 (0x3c07 bytes) 2024-09-15T00:09:22.646 INFO:teuthology.orchestra.run.smithi092.stdout:1420 read 0xa9c3800 thru 0xa9cf4ca (0xbccb bytes) 2024-09-15T00:09:22.646 INFO:teuthology.orchestra.run.smithi092.stdout:1421 read 0x20f2400 thru 0x20fff80 (0xdb81 bytes) 2024-09-15T00:09:22.646 INFO:teuthology.orchestra.run.smithi092.stdout:1422 write 0x1ea1400 thru 0x1ea909e (0x7c9f bytes) 2024-09-15T00:09:22.646 INFO:teuthology.orchestra.run.smithi092.stdout:1423 punch from 0x4c24e00 to 0x4c30c00, (0xbe00 bytes) 2024-09-15T00:09:22.646 INFO:teuthology.orchestra.run.smithi092.stdout:1425 read 0x21e1000 thru 0x21e710b (0x610c bytes) 2024-09-15T00:09:22.646 INFO:teuthology.orchestra.run.smithi092.stdout:1429 write 0x22f0e00 thru 0x22f103e (0x23f bytes) 2024-09-15T00:09:22.647 INFO:teuthology.orchestra.run.smithi092.stdout:1430 read 0xa9c7600 thru 0xa9d73b8 (0xfdb9 bytes) 2024-09-15T00:09:22.647 INFO:teuthology.orchestra.run.smithi092.stdout:1431 punch from 0x638fa00 to 0x6391400, (0x1a00 bytes) 2024-09-15T00:09:22.647 INFO:teuthology.orchestra.run.smithi092.stdout:1432 write 0x8e86400 thru 0x8e90a4e (0xa64f bytes) 2024-09-15T00:09:22.650 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:22.650 INFO:teuthology.orchestra.run.smithi092.stdout:1433 trunc from 0xab11200 to 0xe0ac400 2024-09-15T00:09:22.683 INFO:teuthology.orchestra.run.smithi092.stdout:1434 trunc from 0xe0ac400 to 0x4e14000 2024-09-15T00:09:22.723 INFO:teuthology.orchestra.run.smithi092.stdout:1435 write 0x6895200 thru 0x689995c (0x475d bytes) 2024-09-15T00:09:22.723 INFO:teuthology.orchestra.run.smithi092.stdout:1437 read 0x5a12000 thru 0x5a1d46c (0xb46d bytes) 2024-09-15T00:09:22.741 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:22.740+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:22.744 INFO:teuthology.orchestra.run.smithi092.stdout:1438 write 0x7fda600 thru 0x7fe97cb (0xf1cc bytes) 2024-09-15T00:09:22.744 INFO:teuthology.orchestra.run.smithi092.stdout:1439 read 0x4f7d400 thru 0x4f8a01b (0xcc1c bytes) 2024-09-15T00:09:22.769 INFO:teuthology.orchestra.run.smithi092.stdout:1440 write 0xc33aa00 thru 0xc344c6d (0xa26e bytes) 2024-09-15T00:09:22.769 INFO:teuthology.orchestra.run.smithi092.stdout:1442 write 0x1471800 thru 0x147ddcc (0xc5cd bytes) 2024-09-15T00:09:22.794 INFO:teuthology.orchestra.run.smithi092.stdout:1444 write 0xe05ae00 thru 0xe05c33f (0x1540 bytes) 2024-09-15T00:09:22.803 INFO:teuthology.orchestra.run.smithi092.stdout:1446 trunc from 0xe05c400 to 0x5dda800 2024-09-15T00:09:22.806 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:22.806 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:22.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:22.827 INFO:teuthology.orchestra.run.smithi092.stdout:1448 read 0x982a00 thru 0x98e024 (0xb625 bytes) 2024-09-15T00:09:22.844 INFO:teuthology.orchestra.run.smithi092.stdout:1450 write 0x85b0600 thru 0x85bef2a (0xe92b bytes) 2024-09-15T00:09:22.844 INFO:teuthology.orchestra.run.smithi092.stdout:1455 read 0x7e80800 thru 0x7e9029c (0xfa9d bytes) 2024-09-15T00:09:22.844 INFO:teuthology.orchestra.run.smithi092.stdout:1461 write 0x34c8400 thru 0x34cc2dd (0x3ede bytes) 2024-09-15T00:09:22.848 INFO:teuthology.orchestra.run.smithi092.stdout:1464 trunc from 0x85bf000 to 0xc044e00 2024-09-15T00:09:22.859 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:22.858+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:22.879 INFO:teuthology.orchestra.run.smithi092.stdout:1465 write 0xcf2ae00 thru 0xcf2b87d (0xa7e bytes) 2024-09-15T00:09:22.879 INFO:teuthology.orchestra.run.smithi092.stdout:1466 punch from 0xb77da00 to 0xb784c00, (0x7200 bytes) 2024-09-15T00:09:22.879 INFO:teuthology.orchestra.run.smithi092.stdout:1468 write 0xee2a00 thru 0xeeadb7 (0x83b8 bytes) 2024-09-15T00:09:22.883 INFO:teuthology.orchestra.run.smithi092.stdout:1469 trunc from 0xcf2ba00 to 0x9aed200 2024-09-15T00:09:22.906 INFO:teuthology.orchestra.run.smithi092.stdout:1473 trunc from 0x9aed200 to 0xb3fbc00 2024-09-15T00:09:22.922 INFO:teuthology.orchestra.run.smithi092.stdout:1474 read 0x26f1c00 thru 0x26ff9ac (0xddad bytes) 2024-09-15T00:09:22.922 INFO:teuthology.orchestra.run.smithi092.stdout:1475 read 0x23a2c00 thru 0x23afcdc (0xd0dd bytes) 2024-09-15T00:09:22.928 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:22.928 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:22.931 INFO:teuthology.orchestra.run.smithi092.stdout:1477 trunc from 0xb3fbc00 to 0x4288000 2024-09-15T00:09:22.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:22.942 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:22.941+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:22.950 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:09:22.951 INFO:teuthology.orchestra.run.smithi092.stdout:1483 write 0x127e400 thru 0x128588d (0x748e bytes) 2024-09-15T00:09:22.951 INFO:teuthology.orchestra.run.smithi092.stdout:1485 punch from 0xdc4c00 to 0xdc6e00, (0x2200 bytes) 2024-09-15T00:09:22.951 INFO:teuthology.orchestra.run.smithi092.stdout:1486 punch from 0x2de8600 to 0x2df5c00, (0xd600 bytes) 2024-09-15T00:09:22.969 INFO:teuthology.orchestra.run.smithi092.stdout:1491 write 0x4901c00 thru 0x490c948 (0xad49 bytes) 2024-09-15T00:09:22.969 INFO:teuthology.orchestra.run.smithi092.stdout:1493 read 0x28c2400 thru 0x28d1a87 (0xf688 bytes) 2024-09-15T00:09:22.971 INFO:teuthology.orchestra.run.smithi092.stdout:1494 trunc from 0x490ca00 to 0x622d600 2024-09-15T00:09:22.988 INFO:teuthology.orchestra.run.smithi092.stdout:1496 read 0x41ab400 thru 0x41b6b24 (0xb725 bytes) 2024-09-15T00:09:22.998 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-15T00:09:22.998 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:09:22.998 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T00:09:22.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:09:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:1499 write 0x9faf800 thru 0x9fb2336 (0x2b37 bytes) 2024-09-15T00:09:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:1501 read 0x8f4e200 thru 0x8f57d2d (0x9b2e bytes) 2024-09-15T00:09:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:1504 read 0x5b09600 thru 0x5b1003d (0x6a3e bytes) 2024-09-15T00:09:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:1505 clone 12 order 25 su 8388608 sc 8 2024-09-15T00:09:23.043 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:23.042+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:23.045 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:23.045 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:23.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:23.144 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:23.143+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:23.197 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:23.197 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:23.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:23.250 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:23.249+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:09:23.293 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 39, 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "4f532d31-28c1-4236-a74e-6392960baf2b", 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2024-09-15T00:08:32.007191+0000", 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2024-09-15T00:09:21.854714+0000", 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2024-09-15T00:08:42.794810+0000", 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2024-09-15T00:08:33.890963+0000", 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2024-09-15T00:09:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 3, 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 6, 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "squid", 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2024-09-15T00:09:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:42.896076+0000", 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "14", 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T00:09:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 6.059999942779541, 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 6.059999942779541, 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.33000001311302185, 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 2, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "rbd", 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:49.173060+0000", 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 8, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 8, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 8, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 8, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 8, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "18", 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 18, 2024-09-15T00:09:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2024-09-15T00:09:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2024-09-15T00:09:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2024-09-15T00:09:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "pool_client.0", 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:58.879058+0000", 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2024-09-15T00:09:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "39", 2024-09-15T00:09:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 13, 2024-09-15T00:09:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 39, 2024-09-15T00:09:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "56600dc2-b95d-47a7-b2b1-5ddc54c08341", 2024-09-15T00:09:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2024-09-15T00:09:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2024-09-15T00:09:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/1440638768", 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/1440638768", 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6823/1440638768", 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6821/1440638768", 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "ea8f0155-8472-4e87-89fd-7716d05065d0", 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/3601510193", 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/3601510193", 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/3601510193", 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/3601510193", 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "357d2d59-66f0-47b8-87c8-4acdfe62a07d", 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 26, 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-15T00:09:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2024-09-15T00:09:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/3222759256", 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/3222759256", 2024-09-15T00:09:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/3222759256", 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/3222759256", 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "22208f56-d8aa-4a56-a42c-5a990962b860", 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6809", 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6811", 2024-09-15T00:09:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6813", 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6816", 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6822", 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6823", 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6818", 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6821", 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6811/675428777", 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6816/675428777", 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6823/675428777", 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6821/675428777", 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "a974629e-9ef0-40a3-94bd-382e0bfa8bb6", 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6808", 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6810", 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6812", 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6814", 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6819", 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6820", 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6815", 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6817", 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6810/3736884138", 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6814/3736884138", 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6820/3736884138", 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6817/3736884138", 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "06b73c2d-466a-4c4e-89fa-31d6eeb1af1d", 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6800", 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6801", 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6802", 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6803", 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6806", 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6807", 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6804", 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6805", 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6801/2235671634", 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6803/2235671634", 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6807/2235671634", 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6805/2235671634", 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.730107+0000", 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.365297+0000", 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.868500+0000", 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.788781+0000", 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.892143+0000", 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.990037+0000", 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.1e", 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 3, 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 4 2024-09-15T00:09:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T00:09:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2024-09-15T00:09:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2024-09-15T00:09:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2024-09-15T00:09:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2024-09-15T00:09:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2024-09-15T00:09:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:09:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:09:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2024-09-15T00:09:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:23.360 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:23.363 INFO:tasks.thrashosds.thrasher:j is {'epoch': 39, 'fsid': '4f532d31-28c1-4236-a74e-6392960baf2b', 'created': '2024-09-15T00:08:32.007191+0000', 'modified': '2024-09-15T00:09:21.854714+0000', 'last_up_change': '2024-09-15T00:08:42.794810+0000', 'last_in_change': '2024-09-15T00:08:33.890963+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:08:42.896076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:08:49.173060+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:08:58.879058+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 39, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '56600dc2-b95d-47a7-b2b1-5ddc54c08341', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 1440638768}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 1440638768}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 1440638768}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 1440638768}]}, 'public_addr': '172.21.15.27:6817/1440638768', 'cluster_addr': '172.21.15.27:6819/1440638768', 'heartbeat_back_addr': '172.21.15.27:6823/1440638768', 'heartbeat_front_addr': '172.21.15.27:6821/1440638768', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ea8f0155-8472-4e87-89fd-7716d05065d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 3601510193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 3601510193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 3601510193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 3601510193}]}, 'public_addr': '172.21.15.27:6809/3601510193', 'cluster_addr': '172.21.15.27:6811/3601510193', 'heartbeat_back_addr': '172.21.15.27:6815/3601510193', 'heartbeat_front_addr': '172.21.15.27:6813/3601510193', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '357d2d59-66f0-47b8-87c8-4acdfe62a07d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 3222759256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 3222759256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 3222759256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 3222759256}]}, 'public_addr': '172.21.15.27:6801/3222759256', 'cluster_addr': '172.21.15.27:6803/3222759256', 'heartbeat_back_addr': '172.21.15.27:6807/3222759256', 'heartbeat_front_addr': '172.21.15.27:6805/3222759256', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '22208f56-d8aa-4a56-a42c-5a990962b860', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6809', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 675428777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6813', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6816', 'nonce': 675428777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 675428777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 675428777}]}, 'public_addr': '172.21.15.79:6811/675428777', 'cluster_addr': '172.21.15.79:6816/675428777', 'heartbeat_back_addr': '172.21.15.79:6823/675428777', 'heartbeat_front_addr': '172.21.15.79:6821/675428777', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a974629e-9ef0-40a3-94bd-382e0bfa8bb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6810', 'nonce': 3736884138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6814', 'nonce': 3736884138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6819', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6820', 'nonce': 3736884138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6815', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 3736884138}]}, 'public_addr': '172.21.15.79:6810/3736884138', 'cluster_addr': '172.21.15.79:6814/3736884138', 'heartbeat_back_addr': '172.21.15.79:6820/3736884138', 'heartbeat_front_addr': '172.21.15.79:6817/3736884138', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '06b73c2d-466a-4c4e-89fa-31d6eeb1af1d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 2235671634}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2235671634}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2235671634}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2235671634}]}, 'public_addr': '172.21.15.79:6801/2235671634', 'cluster_addr': '172.21.15.79:6803/2235671634', 'heartbeat_back_addr': '172.21.15.79:6807/2235671634', 'heartbeat_front_addr': '172.21.15.79:6805/2235671634', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.730107+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.365297+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.868500+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.788781+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.892143+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.990037+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1e', 'mappings': [{'from': 3, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:09:23.364 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:09:23.364 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:23.360+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:23.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:23.463 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:23.461+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:23.506 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:23.506 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:23.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:23.552 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:23.605 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:23.604+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:09:23.694 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:09:23.705 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-15T00:09:22.858711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":322776702,"num_objects":308,"num_object_clones":68,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":61,"num_read":49265,"num_read_kb":237540,"num_write":3373,"num_write_kb":56469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2636,"ondisk_log_size":2636,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":200508,"kb_used_data":13100,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566030532,"statfs":{"total":579820584960,"available":579615264768,"internally_reserved":0,"allocated":13414400,"data_stored":77551273,"data_compressed":948216,"data_compressed_allocated":4988928,"data_compressed_original":71404032,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":158507209,"num_objects":74,"num_object_clones":17,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":12,"num_read":21570,"num_read_kb":124456,"num_write":1601,"num_write_kb":33207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.016368"},"pg_stats":[{"pgid":"3.1f","version":"35'132","reported_seq":2783,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.839723+0000","last_change":"2024-09-15T00:09:01.970297+0000","last_active":"2024-09-15T00:09:17.839723+0000","last_peered":"2024-09-15T00:09:17.839723+0000","last_clean":"2024-09-15T00:09:17.839723+0000","last_became_active":"2024-09-15T00:08:59.945793+0000","last_became_peered":"2024-09-15T00:08:59.945793+0000","last_unstale":"2024-09-15T00:09:17.839723+0000","last_undegraded":"2024-09-15T00:09:17.839723+0000","last_fullsized":"2024-09-15T00:09:17.839723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:51:51.139184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075174400000000004,"stat_sum":{"num_bytes":16279056,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":3590,"num_read_kb":13486,"num_write":120,"num_write_kb":802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"36'34","reported_seq":997,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.848198+0000","last_change":"2024-09-15T00:09:06.797902+0000","last_active":"2024-09-15T00:09:17.848198+0000","last_peered":"2024-09-15T00:09:17.848198+0000","last_clean":"2024-09-15T00:09:17.848198+0000","last_became_active":"2024-09-15T00:09:06.795735+0000","last_became_peered":"2024-09-15T00:09:06.795735+0000","last_unstale":"2024-09-15T00:09:17.848198+0000","last_undegraded":"2024-09-15T00:09:17.848198+0000","last_fullsized":"2024-09-15T00:09:17.848198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:21:50.392657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031880599999999998,"stat_sum":{"num_bytes":13802026,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":925,"num_read_kb":7275,"num_write":35,"num_write_kb":1482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"35'129","reported_seq":1177,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.845133+0000","last_change":"2024-09-15T00:09:01.965351+0000","last_active":"2024-09-15T00:09:17.845133+0000","last_peered":"2024-09-15T00:09:17.845133+0000","last_clean":"2024-09-15T00:09:17.845133+0000","last_became_active":"2024-09-15T00:08:59.947317+0000","last_became_peered":"2024-09-15T00:08:59.947317+0000","last_unstale":"2024-09-15T00:09:17.845133+0000","last_undegraded":"2024-09-15T00:09:17.845133+0000","last_fullsized":"2024-09-15T00:09:17.845133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:53:42.423288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001450004,"stat_sum":{"num_bytes":11625511,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":2188,"num_read_kb":2137,"num_write":133,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"35'151","reported_seq":2387,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.845126+0000","last_change":"2024-09-15T00:09:01.964719+0000","last_active":"2024-09-15T00:09:17.845126+0000","last_peered":"2024-09-15T00:09:17.845126+0000","last_clean":"2024-09-15T00:09:17.845126+0000","last_became_active":"2024-09-15T00:08:59.945681+0000","last_became_peered":"2024-09-15T00:08:59.945681+0000","last_unstale":"2024-09-15T00:09:17.845126+0000","last_undegraded":"2024-09-15T00:09:17.845126+0000","last_fullsized":"2024-09-15T00:09:17.845126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:23:41.041074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090320299999999999,"stat_sum":{"num_bytes":17616933,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2723,"num_read_kb":15414,"num_write":244,"num_write_kb":3544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"35'73","reported_seq":2708,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.839574+0000","last_change":"2024-09-15T00:09:01.969947+0000","last_active":"2024-09-15T00:09:17.839574+0000","last_peered":"2024-09-15T00:09:17.839574+0000","last_clean":"2024-09-15T00:09:17.839574+0000","last_became_active":"2024-09-15T00:08:59.945617+0000","last_became_peered":"2024-09-15T00:08:59.945617+0000","last_unstale":"2024-09-15T00:09:17.839574+0000","last_undegraded":"2024-09-15T00:09:17.839574+0000","last_fullsized":"2024-09-15T00:09:17.839574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:30:10.868802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048451499999999998,"stat_sum":{"num_bytes":16324644,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":2779,"num_read_kb":15247,"num_write":123,"num_write_kb":1366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"35'92","reported_seq":3860,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.863732+0000","last_change":"2024-09-15T00:09:01.961084+0000","last_active":"2024-09-15T00:09:17.863732+0000","last_peered":"2024-09-15T00:09:17.863732+0000","last_clean":"2024-09-15T00:09:17.863732+0000","last_became_active":"2024-09-15T00:08:59.946828+0000","last_became_peered":"2024-09-15T00:08:59.946828+0000","last_unstale":"2024-09-15T00:09:17.863732+0000","last_undegraded":"2024-09-15T00:09:17.863732+0000","last_fullsized":"2024-09-15T00:09:17.863732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:18:05.195623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085056300000000004,"stat_sum":{"num_bytes":10561079,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4721,"num_read_kb":15304,"num_write":81,"num_write_kb":4049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"36'83","reported_seq":839,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.847085+0000","last_change":"2024-09-15T00:09:01.958081+0000","last_active":"2024-09-15T00:09:17.847085+0000","last_peered":"2024-09-15T00:09:17.847085+0000","last_clean":"2024-09-15T00:09:17.847085+0000","last_became_active":"2024-09-15T00:08:59.942052+0000","last_became_peered":"2024-09-15T00:08:59.942052+0000","last_unstale":"2024-09-15T00:09:17.847085+0000","last_undegraded":"2024-09-15T00:09:17.847085+0000","last_fullsized":"2024-09-15T00:09:17.847085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:59:58.705513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055705900000000005,"stat_sum":{"num_bytes":6432256,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1183,"num_read_kb":2028,"num_write":94,"num_write_kb":904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"35'151","reported_seq":2787,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.863744+0000","last_change":"2024-09-15T00:09:01.978690+0000","last_active":"2024-09-15T00:09:17.863744+0000","last_peered":"2024-09-15T00:09:17.863744+0000","last_clean":"2024-09-15T00:09:17.863744+0000","last_became_active":"2024-09-15T00:08:59.941686+0000","last_became_peered":"2024-09-15T00:08:59.941686+0000","last_unstale":"2024-09-15T00:09:17.863744+0000","last_undegraded":"2024-09-15T00:09:17.863744+0000","last_fullsized":"2024-09-15T00:09:17.863744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:41:07.415531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000308933,"stat_sum":{"num_bytes":14113792,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":3787,"num_read_kb":20015,"num_write":140,"num_write_kb":4762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"35'69","reported_seq":2288,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.839794+0000","last_change":"2024-09-15T00:09:01.970454+0000","last_active":"2024-09-15T00:09:17.839794+0000","last_peered":"2024-09-15T00:09:17.839794+0000","last_clean":"2024-09-15T00:09:17.839794+0000","last_became_active":"2024-09-15T00:08:59.946139+0000","last_became_peered":"2024-09-15T00:08:59.946139+0000","last_unstale":"2024-09-15T00:09:17.839794+0000","last_undegraded":"2024-09-15T00:09:17.839794+0000","last_fullsized":"2024-09-15T00:09:17.839794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:17.221047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084960000000000005,"stat_sum":{"num_bytes":7237120,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":2676,"num_read_kb":8563,"num_write":48,"num_write_kb":1025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"35'51","reported_seq":1691,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.863789+0000","last_change":"2024-09-15T00:09:01.965029+0000","last_active":"2024-09-15T00:09:17.863789+0000","last_peered":"2024-09-15T00:09:17.863789+0000","last_clean":"2024-09-15T00:09:17.863789+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:09:17.863789+0000","last_undegraded":"2024-09-15T00:09:17.863789+0000","last_fullsized":"2024-09-15T00:09:17.863789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:58:53.352276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062487,"stat_sum":{"num_bytes":19466752,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1605,"num_read_kb":7506,"num_write":79,"num_write_kb":2844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"36'82","reported_seq":974,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.858776+0000","last_change":"2024-09-15T00:09:01.953424+0000","last_active":"2024-09-15T00:09:17.858776+0000","last_peered":"2024-09-15T00:09:17.858776+0000","last_clean":"2024-09-15T00:09:17.858776+0000","last_became_active":"2024-09-15T00:08:59.941565+0000","last_became_peered":"2024-09-15T00:08:59.941565+0000","last_unstale":"2024-09-15T00:09:17.858776+0000","last_undegraded":"2024-09-15T00:09:17.858776+0000","last_fullsized":"2024-09-15T00:09:17.858776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:42:08.614884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044338899999999998,"stat_sum":{"num_bytes":2502144,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":936,"num_read_kb":1336,"num_write":89,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"35'32","reported_seq":442,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.843364+0000","last_change":"2024-09-15T00:09:01.953831+0000","last_active":"2024-09-15T00:09:17.843364+0000","last_peered":"2024-09-15T00:09:17.843364+0000","last_clean":"2024-09-15T00:09:17.843364+0000","last_became_active":"2024-09-15T00:08:59.942776+0000","last_became_peered":"2024-09-15T00:08:59.942776+0000","last_unstale":"2024-09-15T00:09:17.843364+0000","last_undegraded":"2024-09-15T00:09:17.843364+0000","last_fullsized":"2024-09-15T00:09:17.843364+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:29:31.906666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079073999999999995,"stat_sum":{"num_bytes":4734992,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":368,"num_read_kb":3377,"num_write":38,"num_write_kb":453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.843488+0000","last_change":"2024-09-15T00:08:52.835701+0000","last_active":"2024-09-15T00:09:17.843488+0000","last_peered":"2024-09-15T00:09:17.843488+0000","last_clean":"2024-09-15T00:09:17.843488+0000","last_became_active":"2024-09-15T00:08:50.969842+0000","last_became_peered":"2024-09-15T00:08:50.969842+0000","last_unstale":"2024-09-15T00:09:17.843488+0000","last_undegraded":"2024-09-15T00:09:17.843488+0000","last_fullsized":"2024-09-15T00:09:17.843488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:00.926947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"35'42","reported_seq":475,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.839692+0000","last_change":"2024-09-15T00:09:01.970380+0000","last_active":"2024-09-15T00:09:17.839692+0000","last_peered":"2024-09-15T00:09:17.839692+0000","last_clean":"2024-09-15T00:09:17.839692+0000","last_became_active":"2024-09-15T00:08:59.946388+0000","last_became_peered":"2024-09-15T00:08:59.946388+0000","last_unstale":"2024-09-15T00:09:17.839692+0000","last_undegraded":"2024-09-15T00:09:17.839692+0000","last_fullsized":"2024-09-15T00:09:17.839692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:06:32.696249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083591999999999998,"stat_sum":{"num_bytes":5643776,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":387,"num_read_kb":2002,"num_write":39,"num_write_kb":513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.847307+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:09:17.847307+0000","last_peered":"2024-09-15T00:09:17.847307+0000","last_clean":"2024-09-15T00:09:17.847307+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:09:17.847307+0000","last_undegraded":"2024-09-15T00:09:17.847307+0000","last_fullsized":"2024-09-15T00:09:17.847307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"35'33","reported_seq":368,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.848277+0000","last_change":"2024-09-15T00:09:01.979030+0000","last_active":"2024-09-15T00:09:17.848277+0000","last_peered":"2024-09-15T00:09:17.848277+0000","last_clean":"2024-09-15T00:09:17.848277+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:09:17.848277+0000","last_undegraded":"2024-09-15T00:09:17.848277+0000","last_fullsized":"2024-09-15T00:09:17.848277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:10:54.550105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059204099999999996,"stat_sum":{"num_bytes":7090734,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":304,"num_read_kb":2268,"num_write":32,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.843505+0000","last_change":"2024-09-15T00:08:52.835997+0000","last_active":"2024-09-15T00:09:17.843505+0000","last_peered":"2024-09-15T00:09:17.843505+0000","last_clean":"2024-09-15T00:09:17.843505+0000","last_became_active":"2024-09-15T00:08:51.070763+0000","last_became_peered":"2024-09-15T00:08:51.070763+0000","last_unstale":"2024-09-15T00:09:17.843505+0000","last_undegraded":"2024-09-15T00:09:17.843505+0000","last_fullsized":"2024-09-15T00:09:17.843505+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:58.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"35'231","reported_seq":1686,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.847144+0000","last_change":"2024-09-15T00:09:01.958574+0000","last_active":"2024-09-15T00:09:17.847144+0000","last_peered":"2024-09-15T00:09:17.847144+0000","last_clean":"2024-09-15T00:09:17.847144+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:09:17.847144+0000","last_undegraded":"2024-09-15T00:09:17.847144+0000","last_fullsized":"2024-09-15T00:09:17.847144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:25:02.137086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070311699999999996,"stat_sum":{"num_bytes":5177421,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":3366,"num_read_kb":5997,"num_write":276,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.847163+0000","last_change":"2024-09-15T00:08:52.842966+0000","last_active":"2024-09-15T00:09:17.847163+0000","last_peered":"2024-09-15T00:09:17.847163+0000","last_clean":"2024-09-15T00:09:17.847163+0000","last_became_active":"2024-09-15T00:08:51.070528+0000","last_became_peered":"2024-09-15T00:08:51.070528+0000","last_unstale":"2024-09-15T00:09:17.847163+0000","last_undegraded":"2024-09-15T00:09:17.847163+0000","last_fullsized":"2024-09-15T00:09:17.847163+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:52:05.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"35'43","reported_seq":919,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.864517+0000","last_change":"2024-09-15T00:09:01.965506+0000","last_active":"2024-09-15T00:09:17.864517+0000","last_peered":"2024-09-15T00:09:17.864517+0000","last_clean":"2024-09-15T00:09:17.864517+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:09:17.864517+0000","last_undegraded":"2024-09-15T00:09:17.864517+0000","last_fullsized":"2024-09-15T00:09:17.864517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:01:50.794871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001375294,"stat_sum":{"num_bytes":3844128,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":855,"num_read_kb":5267,"num_write":43,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"35'44","reported_seq":1145,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.847993+0000","last_change":"2024-09-15T00:09:01.960514+0000","last_active":"2024-09-15T00:09:17.847993+0000","last_peered":"2024-09-15T00:09:17.847993+0000","last_clean":"2024-09-15T00:09:17.847993+0000","last_became_active":"2024-09-15T00:08:59.944404+0000","last_became_peered":"2024-09-15T00:08:59.944404+0000","last_unstale":"2024-09-15T00:09:17.847993+0000","last_undegraded":"2024-09-15T00:09:17.847993+0000","last_fullsized":"2024-09-15T00:09:17.847993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:09.583198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037381900000000001,"stat_sum":{"num_bytes":5989415,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":1072,"num_read_kb":7938,"num_write":70,"num_write_kb":883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":57,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.845043+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:09:17.845043+0000","last_peered":"2024-09-15T00:09:17.845043+0000","last_clean":"2024-09-15T00:09:17.845043+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:09:17.845043+0000","last_undegraded":"2024-09-15T00:09:17.845043+0000","last_fullsized":"2024-09-15T00:09:17.845043+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"36'191","reported_seq":548,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.867881+0000","last_change":"2024-09-15T00:09:01.958075+0000","last_active":"2024-09-15T00:09:17.867881+0000","last_peered":"2024-09-15T00:09:17.867881+0000","last_clean":"2024-09-15T00:09:17.867881+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:09:17.867881+0000","last_undegraded":"2024-09-15T00:09:17.867881+0000","last_fullsized":"2024-09-15T00:09:17.867881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:31:29.327623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000567013,"stat_sum":{"num_bytes":3299988,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1268,"num_read_kb":3148,"num_write":340,"num_write_kb":1621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.848128+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:09:17.848128+0000","last_peered":"2024-09-15T00:09:17.848128+0000","last_clean":"2024-09-15T00:09:17.848128+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:09:17.848128+0000","last_undegraded":"2024-09-15T00:09:17.848128+0000","last_fullsized":"2024-09-15T00:09:17.848128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"35'61","reported_seq":626,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.839576+0000","last_change":"2024-09-15T00:09:01.969727+0000","last_active":"2024-09-15T00:09:17.839576+0000","last_peered":"2024-09-15T00:09:17.839576+0000","last_clean":"2024-09-15T00:09:17.839576+0000","last_became_active":"2024-09-15T00:08:59.942545+0000","last_became_peered":"2024-09-15T00:08:59.942545+0000","last_unstale":"2024-09-15T00:09:17.839576+0000","last_undegraded":"2024-09-15T00:09:17.839576+0000","last_fullsized":"2024-09-15T00:09:17.839576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:59:03.873541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000355144,"stat_sum":{"num_bytes":6073392,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":733,"num_read_kb":5257,"num_write":93,"num_write_kb":993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.843277+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:09:17.843277+0000","last_peered":"2024-09-15T00:09:17.843277+0000","last_clean":"2024-09-15T00:09:17.843277+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:09:17.843277+0000","last_undegraded":"2024-09-15T00:09:17.843277+0000","last_fullsized":"2024-09-15T00:09:17.843277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"35'46","reported_seq":710,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.847158+0000","last_change":"2024-09-15T00:09:01.958272+0000","last_active":"2024-09-15T00:09:17.847158+0000","last_peered":"2024-09-15T00:09:17.847158+0000","last_clean":"2024-09-15T00:09:17.847158+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:09:17.847158+0000","last_undegraded":"2024-09-15T00:09:17.847158+0000","last_fullsized":"2024-09-15T00:09:17.847158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:03:04.464540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056893999999999998,"stat_sum":{"num_bytes":4903440,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":641,"num_read_kb":5710,"num_write":51,"num_write_kb":3141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":56,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.845096+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:09:17.845096+0000","last_peered":"2024-09-15T00:09:17.845096+0000","last_clean":"2024-09-15T00:09:17.845096+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:09:17.845096+0000","last_undegraded":"2024-09-15T00:09:17.845096+0000","last_fullsized":"2024-09-15T00:09:17.845096+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":206,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.843284+0000","last_change":"2024-09-15T00:08:44.804415+0000","last_active":"2024-09-15T00:09:17.843284+0000","last_peered":"2024-09-15T00:09:17.843284+0000","last_clean":"2024-09-15T00:09:17.843284+0000","last_became_active":"2024-09-15T00:08:44.804011+0000","last_became_peered":"2024-09-15T00:08:44.804011+0000","last_unstale":"2024-09-15T00:09:17.843284+0000","last_undegraded":"2024-09-15T00:09:17.843284+0000","last_fullsized":"2024-09-15T00:09:17.843284+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:12.388733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"35'70","reported_seq":1867,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.848320+0000","last_change":"2024-09-15T00:09:01.978947+0000","last_active":"2024-09-15T00:09:17.848320+0000","last_peered":"2024-09-15T00:09:17.848320+0000","last_clean":"2024-09-15T00:09:17.848320+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:09:17.848320+0000","last_undegraded":"2024-09-15T00:09:17.848320+0000","last_fullsized":"2024-09-15T00:09:17.848320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:03:14.872602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051427500000000002,"stat_sum":{"num_bytes":33220624,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":2080,"num_read_kb":16485,"num_write":105,"num_write_kb":5382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"35'61","reported_seq":2698,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.863815+0000","last_change":"2024-09-15T00:09:01.960508+0000","last_active":"2024-09-15T00:09:17.863815+0000","last_peered":"2024-09-15T00:09:17.863815+0000","last_clean":"2024-09-15T00:09:17.863815+0000","last_became_active":"2024-09-15T00:08:59.945869+0000","last_became_peered":"2024-09-15T00:08:59.945869+0000","last_unstale":"2024-09-15T00:09:17.863815+0000","last_undegraded":"2024-09-15T00:09:17.863815+0000","last_fullsized":"2024-09-15T00:09:17.863815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:51:44.466909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031568600000000003,"stat_sum":{"num_bytes":22127616,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":2614,"num_read_kb":20839,"num_write":87,"num_write_kb":4919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"35'39","reported_seq":227,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.843307+0000","last_change":"2024-09-15T00:09:01.953327+0000","last_active":"2024-09-15T00:09:17.843307+0000","last_peered":"2024-09-15T00:09:17.843307+0000","last_clean":"2024-09-15T00:09:17.843307+0000","last_became_active":"2024-09-15T00:08:59.942796+0000","last_became_peered":"2024-09-15T00:08:59.942796+0000","last_unstale":"2024-09-15T00:09:17.843307+0000","last_undegraded":"2024-09-15T00:09:17.843307+0000","last_fullsized":"2024-09-15T00:09:17.843307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:53:34.967513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035268300000000001,"stat_sum":{"num_bytes":1576083,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":169,"num_read_kb":1081,"num_write":37,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"35'54","reported_seq":877,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.847147+0000","last_change":"2024-09-15T00:09:01.958373+0000","last_active":"2024-09-15T00:09:17.847147+0000","last_peered":"2024-09-15T00:09:17.847147+0000","last_clean":"2024-09-15T00:09:17.847147+0000","last_became_active":"2024-09-15T00:08:59.941757+0000","last_became_peered":"2024-09-15T00:08:59.941757+0000","last_unstale":"2024-09-15T00:09:17.847147+0000","last_undegraded":"2024-09-15T00:09:17.847147+0000","last_fullsized":"2024-09-15T00:09:17.847147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:17:50.501344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071565300000000001,"stat_sum":{"num_bytes":12695568,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":787,"num_read_kb":7238,"num_write":52,"num_write_kb":377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"35'118","reported_seq":746,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.845130+0000","last_change":"2024-09-15T00:09:01.964756+0000","last_active":"2024-09-15T00:09:17.845130+0000","last_peered":"2024-09-15T00:09:17.845130+0000","last_clean":"2024-09-15T00:09:17.845130+0000","last_became_active":"2024-09-15T00:08:59.946686+0000","last_became_peered":"2024-09-15T00:08:59.946686+0000","last_unstale":"2024-09-15T00:09:17.845130+0000","last_undegraded":"2024-09-15T00:09:17.845130+0000","last_fullsized":"2024-09-15T00:09:17.845130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:58:32.749763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092954400000000003,"stat_sum":{"num_bytes":2998820,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":989,"num_read_kb":3621,"num_write":169,"num_write_kb":1804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"35'18","reported_seq":382,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.863735+0000","last_change":"2024-09-15T00:09:01.953837+0000","last_active":"2024-09-15T00:09:17.863735+0000","last_peered":"2024-09-15T00:09:17.863735+0000","last_clean":"2024-09-15T00:09:17.863735+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:09:17.863735+0000","last_undegraded":"2024-09-15T00:09:17.863735+0000","last_fullsized":"2024-09-15T00:09:17.863735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:06:03.644586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078421999999999997,"stat_sum":{"num_bytes":7747111,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":327,"num_read_kb":1452,"num_write":16,"num_write_kb":239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"35'44","reported_seq":1808,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.863942+0000","last_change":"2024-09-15T00:09:01.965099+0000","last_active":"2024-09-15T00:09:17.863942+0000","last_peered":"2024-09-15T00:09:17.863942+0000","last_clean":"2024-09-15T00:09:17.863942+0000","last_became_active":"2024-09-15T00:08:59.945750+0000","last_became_peered":"2024-09-15T00:08:59.945750+0000","last_unstale":"2024-09-15T00:09:17.863942+0000","last_undegraded":"2024-09-15T00:09:17.863942+0000","last_fullsized":"2024-09-15T00:09:17.863942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:36:38.850758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010923459999999999,"stat_sum":{"num_bytes":20629552,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":1755,"num_read_kb":11702,"num_write":49,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"35'27","reported_seq":182,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.845245+0000","last_change":"2024-09-15T00:09:01.965290+0000","last_active":"2024-09-15T00:09:17.845245+0000","last_peered":"2024-09-15T00:09:17.845245+0000","last_clean":"2024-09-15T00:09:17.845245+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:09:17.845245+0000","last_undegraded":"2024-09-15T00:09:17.845245+0000","last_fullsized":"2024-09-15T00:09:17.845245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:41:37.388665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001242525,"stat_sum":{"num_bytes":3276816,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":112,"num_read_kb":464,"num_write":26,"num_write_kb":228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"35'47","reported_seq":467,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.864790+0000","last_change":"2024-09-15T00:09:01.964202+0000","last_active":"2024-09-15T00:09:17.864790+0000","last_peered":"2024-09-15T00:09:17.864790+0000","last_clean":"2024-09-15T00:09:17.864790+0000","last_became_active":"2024-09-15T00:08:59.945540+0000","last_became_peered":"2024-09-15T00:08:59.945540+0000","last_unstale":"2024-09-15T00:09:17.864790+0000","last_undegraded":"2024-09-15T00:09:17.864790+0000","last_fullsized":"2024-09-15T00:09:17.864790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:41:57.131235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041955600000000002,"stat_sum":{"num_bytes":6950912,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":384,"num_read_kb":5171,"num_write":83,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"35'44","reported_seq":722,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.843277+0000","last_change":"2024-09-15T00:09:01.953573+0000","last_active":"2024-09-15T00:09:17.843277+0000","last_peered":"2024-09-15T00:09:17.843277+0000","last_clean":"2024-09-15T00:09:17.843277+0000","last_became_active":"2024-09-15T00:08:59.942379+0000","last_became_peered":"2024-09-15T00:08:59.942379+0000","last_unstale":"2024-09-15T00:09:17.843277+0000","last_undegraded":"2024-09-15T00:09:17.843277+0000","last_fullsized":"2024-09-15T00:09:17.843277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:22:28.639970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067808099999999997,"stat_sum":{"num_bytes":2785280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":631,"num_read_kb":2910,"num_write":45,"num_write_kb":663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"35'188","reported_seq":1987,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.863835+0000","last_change":"2024-09-15T00:09:01.969823+0000","last_active":"2024-09-15T00:09:17.863835+0000","last_peered":"2024-09-15T00:09:17.863835+0000","last_clean":"2024-09-15T00:09:17.863835+0000","last_became_active":"2024-09-15T00:08:59.946308+0000","last_became_peered":"2024-09-15T00:08:59.946308+0000","last_unstale":"2024-09-15T00:09:17.863835+0000","last_undegraded":"2024-09-15T00:09:17.863835+0000","last_fullsized":"2024-09-15T00:09:17.863835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:39:57.870184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041010299999999999,"stat_sum":{"num_bytes":20576864,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":2562,"num_read_kb":14819,"num_write":342,"num_write_kb":3972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"36'36","reported_seq":704,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T00:09:17.864487+0000","last_change":"2024-09-15T00:09:01.953644+0000","last_active":"2024-09-15T00:09:17.864487+0000","last_peered":"2024-09-15T00:09:17.864487+0000","last_clean":"2024-09-15T00:09:17.864487+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:09:17.864487+0000","last_undegraded":"2024-09-15T00:09:17.864487+0000","last_fullsized":"2024-09-15T00:09:17.864487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:00:35.055325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061215200000000003,"stat_sum":{"num_bytes":4882470,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":632,"num_read_kb":2383,"num_write":33,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":322186315,"num_objects":304,"num_object_clones":68,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":61,"num_read":49149,"num_read_kb":237440,"num_write":3212,"num_write_kb":54051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10657792,"data_stored":76528790,"data_compressed":1013584,"data_compressed_allocated":4734976,"data_compressed_original":71433216,"omap_allocated":0,"internal_metadata":0},"log_size":2516,"ondisk_log_size":2516,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640266,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1970176,"data_stored":5952865,"data_compressed":169927,"data_compressed_allocated":638976,"data_compressed_original":4996096,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640266,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2699264,"data_stored":16319555,"data_compressed":157617,"data_compressed_allocated":909312,"data_compressed_original":14942208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640267,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1691648,"data_stored":6000088,"data_compressed":156788,"data_compressed_allocated":471040,"data_compressed_original":5132800,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640266,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33964,"kb_used_data":2780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337876,"statfs":{"total":96636764160,"available":96601985024,"internally_reserved":0,"allocated":2846720,"data_stored":23374308,"data_compressed":200565,"data_compressed_allocated":1363968,"data_compressed_original":22286336,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640266,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1871872,"data_stored":12485058,"data_compressed":99711,"data_compressed_allocated":626688,"data_compressed_original":11598848,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640267,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":13419399,"data_compressed":163608,"data_compressed_allocated":978944,"data_compressed_original":12447744,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1875968,"data_stored":12649759,"data_compressed":154812,"data_compressed_allocated":925696,"data_compressed_original":11841536,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1499136,"data_stored":12364662,"data_compressed":99912,"data_compressed_allocated":638976,"data_compressed_original":11623424,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2236416,"data_stored":23403408,"data_compressed":226627,"data_compressed_allocated":1224704,"data_compressed_original":22544896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1245184,"data_stored":5251440,"data_compressed":148193,"data_compressed_allocated":430080,"data_compressed_original":4551168,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2355200,"data_stored":16535035,"data_compressed":167670,"data_compressed_allocated":954368,"data_compressed_original":15310848,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1445888,"data_stored":6324486,"data_compressed":216370,"data_compressed_allocated":561152,"data_compressed_original":5561344,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:09:23.707 INFO:tasks.thrashosds.thrasher:Setting 3.1d to [1, 0, 4, 2] 2024-09-15T00:09:23.707 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1d', '1', '0', '4', '2'] 2024-09-15T00:09:23.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1d 1 0 4 2 2024-09-15T00:09:23.708 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:23.704+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:23.709 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:23.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:23.728 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:23.728 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:23.745 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:23.762 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:23.762 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:23.780 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:23.780 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:23.797 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:23.797 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:23.806 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:23.805+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:23.813 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:23.814 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:23.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:09:23.907 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:23.906+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:23.936 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:23.936 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:23.936 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:23.936 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:23.937 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:23.944 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone11 from 0x9fb2400 (overlap 0x3a3c00) to 0x0 2024-09-15T00:09:23.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:24.029 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:24.028+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:24.092 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1142 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2070878030000003, 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045190363999999997, 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1142 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882756+0000", 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-15T00:09:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882773+0000", 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6872999999999999e-05 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882780+0000", 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4240000000000004e-06 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882787+0000", 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8510000000000001e-06 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927799+0000", 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045011974000000003 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927843+0000", 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4270999999999997e-05 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927946+0000", 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102581 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2070391110000003, 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:09:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:09:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2070004809999997, 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2069745440000004, 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2069584759999996, 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:09:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:09:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2068173809999996, 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:09:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:09:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2060456259999999, 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2059605949999996, 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:09:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:09:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2059255789999996, 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:09:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2058999760000004, 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:09:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2057257100000003, 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.205494571, 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:09:24.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1157 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2047131330000003, 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045184281, 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1157 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885132+0000", 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885139+0000", 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0350000000000004e-06 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885153+0000", 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3515e-05 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930177+0000", 2024-09-15T00:09:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045024495999999997 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930249+0000", 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2072999999999999e-05 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930314+0000", 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4826999999999994e-05 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2046826270000004, 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:09:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:09:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2046028870000001, 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:09:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:09:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2043978620000004, 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:09:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:09:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2041863289999997, 2024-09-15T00:09:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:09:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2016303219999998, 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.1996472159999998, 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:09:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.1996276449999996, 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:24.180 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:09:24.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:09:24.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:24.185 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:24.185 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:24.186 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:24.187 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:24.185+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:24.287 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:24.286+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:24.293 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:24.294 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:24.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:24.388 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:24.386+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:24.444 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-15T00:09:24.455 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:24.455 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:24.465 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:24.489 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:24.487+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:24.589 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:24.588+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:24.611 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8827 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.152919+0000", 2024-09-15T00:09:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.455048202, 2024-09-15T00:09:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051030585000000003, 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8827 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152919+0000", 2024-09-15T00:09:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152919+0000", 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152920+0000", 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0800000000000001e-07 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152921+0000", 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.514e-06 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152931+0000", 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0146000000000001e-05 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152934+0000", 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1099999999999999e-06 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.197686+0000", 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044752029999999998 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.197764+0000", 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7097999999999999e-05 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.198025+0000", 2024-09-15T00:09:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000261754 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.201583+0000", 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035579750000000001 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203867+0000", 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002284075 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203921+0000", 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3946000000000002e-05 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203950+0000", 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8728999999999999e-05 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8828 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.152939+0000", 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4550286030000001, 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051100758000000003, 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8828 2024-09-15T00:09:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152939+0000", 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152939+0000", 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152939+0000", 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5199999999999998e-07 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152941+0000", 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0439999999999998e-06 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152945+0000", 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4640000000000002e-06 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152949+0000", 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1400000000000002e-06 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.198569+0000", 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.045619932000000002 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.198655+0000", 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6416999999999996e-05 2024-09-15T00:09:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.198905+0000", 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024978100000000003 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203104+0000", 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0041986410000000004 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203980+0000", 2024-09-15T00:09:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087610800000000001 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204010+0000", 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0378999999999998e-05 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204040+0000", 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9600000000000001e-05 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8829 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.152953+0000", 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4550141860000001, 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053638358999999997, 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8829 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152953+0000", 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152953+0000", 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152954+0000", 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.05e-07 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152955+0000", 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5600000000000001e-06 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152958+0000", 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0149999999999999e-06 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152962+0000", 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2039999999999999e-06 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.199441+0000", 2024-09-15T00:09:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.046478763999999999 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.199520+0000", 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8741999999999999e-05 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.199790+0000", 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000269728 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203810+0000", 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040201669999999998 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206495+0000", 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002684723 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206549+0000", 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4700000000000001e-05 2024-09-15T00:09:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206592+0000", 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2551e-05 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8830 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.152969+0000", 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4549989870000002, 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053767113999999998, 2024-09-15T00:09:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8830 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152969+0000", 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152969+0000", 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152969+0000", 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4200000000000002e-07 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152972+0000", 2024-09-15T00:09:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5900000000000002e-06 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152977+0000", 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.728e-06 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152980+0000", 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8399999999999999e-06 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.200133+0000", 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.047152573000000003 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.200230+0000", 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7162000000000002e-05 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.200633+0000", 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040294700000000002 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203818+0000", 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031854129999999998 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206672+0000", 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002853469 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206708+0000", 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6158999999999997e-05 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206736+0000", 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7891000000000001e-05 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8831 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.152985+0000", 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.454982963, 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053892043000000001, 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8831 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152985+0000", 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152985+0000", 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152985+0000", 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2499999999999999e-07 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152987+0000", 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.646e-06 2024-09-15T00:09:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152990+0000", 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6540000000000001e-06 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.152993+0000", 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6060000000000001e-06 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.201622+0000", 2024-09-15T00:09:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.048629393999999999 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.201674+0000", 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1921e-05 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.201881+0000", 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020730400000000001 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204048+0000", 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021660529999999998 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206787+0000", 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0027391609999999999 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206839+0000", 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2364000000000002e-05 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206877+0000", 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7715000000000001e-05 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8832 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4549620839999999, 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056083069999999999, 2024-09-15T00:09:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8832 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2099999999999998e-07 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153008+0000", 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.187e-06 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153012+0000", 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8909999999999997e-06 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153015+0000", 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8559999999999998e-06 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202369+0000", 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.049354393000000003 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202431+0000", 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2235000000000003e-05 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202749+0000", 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00031714299999999998 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204050+0000", 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00130164 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209033+0000", 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049826590000000004 2024-09-15T00:09:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209063+0000", 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0176999999999999e-05 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209089+0000", 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5568e-05 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8833 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4549476050000001, 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056156170999999998, 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8833 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1299999999999999e-07 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153022+0000", 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5829999999999999e-06 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153025+0000", 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.022e-06 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153029+0000", 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8500000000000004e-06 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203131+0000", 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.05010212 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203216+0000", 2024-09-15T00:09:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.543e-05 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203493+0000", 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027699000000000002 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206606+0000", 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003112315 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209120+0000", 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002514863 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209149+0000", 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8685000000000001e-05 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209176+0000", 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7100000000000001e-05 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8834 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4549360629999999, 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056218377999999999, 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8834 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.16e-07 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153034+0000", 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.844e-06 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153039+0000", 2024-09-15T00:09:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8369999999999996e-06 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153041+0000", 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5620000000000002e-06 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204094+0000", 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051053378000000003 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204144+0000", 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9907999999999997e-05 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204342+0000", 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000197596 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206613+0000", 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022710579999999998 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209204+0000", 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0025907780000000002 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209230+0000", 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6041999999999999e-05 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209250+0000", 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0159000000000001e-05 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8835 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4549236529999998, 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060077884999999998, 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8835 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.16e-07 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153046+0000", 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T00:09:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153049+0000", 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153051+0000", 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.52e-06 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204715+0000", 2024-09-15T00:09:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051664032999999998 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204804+0000", 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8383999999999999e-05 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205194+0000", 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039034399999999999 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206890+0000", 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016954680000000001 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213068+0000", 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006178411 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213097+0000", 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9433000000000001e-05 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213122+0000", 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4553999999999999e-05 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8836 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4549132679999999, 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060148357, 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8836 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153055+0000", 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0100000000000001e-07 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153056+0000", 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6220000000000001e-06 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153060+0000", 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5499999999999999e-06 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153062+0000", 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6510000000000001e-06 2024-09-15T00:09:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205724+0000", 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.052661396999999999 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205802+0000", 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8668e-05 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206109+0000", 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030663400000000002 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.208482+0000", 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023725019999999999 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213151+0000", 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046689660000000001 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213181+0000", 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0324999999999998e-05 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213203+0000", 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1841e-05 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8837 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.454902395, 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060386983999999998, 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8837 2024-09-15T00:09:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1500000000000001e-07 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153067+0000", 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.764e-06 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153070+0000", 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.869e-06 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153073+0000", 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.548e-06 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206907+0000", 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053834290999999999 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206953+0000", 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5648000000000001e-05 2024-09-15T00:09:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207282+0000", 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032939999999999998 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209097+0000", 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018149170000000001 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213388+0000", 2024-09-15T00:09:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042910989999999996 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213430+0000", 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2465000000000001e-05 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213452+0000", 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1767999999999999e-05 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8838 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4548921529999999, 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060498379999999997, 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8838 2024-09-15T00:09:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.04e-07 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153077+0000", 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.389e-06 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153080+0000", 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8499999999999998e-06 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153083+0000", 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.565e-06 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207872+0000", 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.054789110000000002 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207926+0000", 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4515000000000002e-05 2024-09-15T00:09:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.208143+0000", 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000216784 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209511+0000", 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013684350000000001 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213494+0000", 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039824889999999996 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213535+0000", 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0865999999999998e-05 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213574+0000", 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9172999999999999e-05 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:10082 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.363504+0000", 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.2444637790000002, 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.04926854, 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 10082 2024-09-15T00:09:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363504+0000", 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363504+0000", 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363504+0000", 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3700000000000001e-07 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363507+0000", 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.627e-06 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363513+0000", 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9560000000000002e-06 2024-09-15T00:09:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363518+0000", 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8040000000000001e-06 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.406911+0000", 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.043392907000000001 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.406960+0000", 2024-09-15T00:09:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9499999999999997e-05 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.407154+0000", 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019361500000000001 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.410633+0000", 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034794330000000001 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.412730+0000", 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002096348 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.412752+0000", 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2804000000000001e-05 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.412772+0000", 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0109e-05 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:10083 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.363525+0000", 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.2444427, 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.050049609000000002, 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 10083 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363525+0000", 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363525+0000", 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363525+0000", 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1600000000000001e-07 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363528+0000", 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.559e-06 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363534+0000", 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3810000000000004e-06 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363539+0000", 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9540000000000003e-06 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.407540+0000", 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044001099000000002 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.407589+0000", 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8479000000000001e-05 2024-09-15T00:09:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.407773+0000", 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018398899999999999 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.410844+0000", 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0030711190000000002 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413492+0000", 2024-09-15T00:09:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026483909999999999 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413539+0000", 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6480000000000002e-05 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413575+0000", 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5642000000000002e-05 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:10084 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.363546+0000", 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.244422176, 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.050120670999999999, 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 10084 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363546+0000", 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363546+0000", 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363546+0000", 2024-09-15T00:09:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0400000000000002e-07 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363548+0000", 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4899999999999999e-06 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363554+0000", 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9150000000000001e-06 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363559+0000", 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6519999999999997e-06 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.408201+0000", 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044642235000000002 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.408247+0000", 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6179e-05 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.408436+0000", 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018908600000000001 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.410872+0000", 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024357509999999999 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413585+0000", 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0027123130000000001 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413630+0000", 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5766999999999997e-05 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413666+0000", 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5879e-05 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:10085 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.363569+0000", 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.2443987540000001, 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.050185727999999999, 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 10085 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363569+0000", 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363569+0000", 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363569+0000", 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-07 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363572+0000", 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.717e-06 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363588+0000", 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5894999999999999e-05 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363595+0000", 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5019999999999998e-06 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.409116+0000", 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.045521508000000002 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.409179+0000", 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3058999999999994e-05 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.409593+0000", 2024-09-15T00:09:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041422099999999999 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.410900+0000", 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013071370000000001 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413675+0000", 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0027746329999999999 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413719+0000", 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3418999999999999e-05 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413755+0000", 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6130000000000001e-05 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:10086 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.363603+0000", 2024-09-15T00:09:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.244364757, 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.050243646000000003, 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 10086 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363603+0000", 2024-09-15T00:09:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363603+0000", 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363603+0000", 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363606+0000", 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8770000000000002e-06 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363613+0000", 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7560000000000002e-06 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363618+0000", 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3689999999999998e-06 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.410038+0000", 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.046419632000000002 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.410086+0000", 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7805000000000001e-05 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.410290+0000", 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020361700000000001 2024-09-15T00:09:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413480+0000", 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031900499999999998 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413764+0000", 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000284357 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413809+0000", 2024-09-15T00:09:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4700000000000002e-05 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413847+0000", 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7954000000000003e-05 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:10087 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.363626+0000", 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.2443416439999999, 2024-09-15T00:09:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053840599000000003, 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 10087 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363626+0000", 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363626+0000", 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363627+0000", 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8599999999999998e-07 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363629+0000", 2024-09-15T00:09:24.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6910000000000002e-06 2024-09-15T00:09:24.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363635+0000", 2024-09-15T00:09:24.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2580000000000001e-06 2024-09-15T00:09:24.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363640+0000", 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6430000000000003e-06 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.410924+0000", 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.047284222000000001 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.410979+0000", 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4216999999999999e-05 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.411345+0000", 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00036671799999999998 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413877+0000", 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0025316890000000002 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.417371+0000", 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003494053 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.417428+0000", 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7454999999999998e-05 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.417467+0000", 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8167000000000002e-05 2024-09-15T00:09:24.721 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:10088 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.363650+0000", 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.2443179569999998, 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053883158, 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.722 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 10088 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363650+0000", 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363650+0000", 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363650+0000", 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7899999999999999e-07 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363653+0000", 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.004e-06 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363662+0000", 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6109999999999994e-06 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363668+0000", 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8250000000000001e-06 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.411918+0000", 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.048250653999999997 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.411993+0000", 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4561999999999995e-05 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.412294+0000", 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030169399999999997 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413907+0000", 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016125810000000001 2024-09-15T00:09:24.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.417474+0000", 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035669870000000002 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.417502+0000", 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7793999999999999e-05 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.417533+0000", 2024-09-15T00:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1066999999999999e-05 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:10089 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.363675+0000", 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.244292636, 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053960112999999997, 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:24.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 10089 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363675+0000", 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363675+0000", 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:24.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363675+0000", 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363678+0000", 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9780000000000001e-06 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363686+0000", 2024-09-15T00:09:24.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3529999999999997e-06 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.363691+0000", 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9860000000000002e-06 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.412804+0000", 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.049113428000000001 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:24.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.412853+0000", 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9106e-05 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.413037+0000", 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000183647 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.415965+0000", 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002928122 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.417542+0000", 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015772399999999999 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.417579+0000", 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6468999999999999e-05 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.417635+0000", 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6363999999999998e-05 2024-09-15T00:09:24.733 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.734 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:24.734 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:24.734 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:24.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:24.736 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:24.734+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:24.836 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:24.835+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:24.867 INFO:teuthology.orchestra.run.smithi027.stderr:set 3.1d pg_upmap_items mapping to [1->0,4->2] 2024-09-15T00:09:24.909 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:24.909 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:24.909 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:24.909 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:24.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:24.937 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:24.936+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:25.039 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:25.038+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:25.065 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:25.065 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:25.065 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:25.066 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:25.066 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:25.076 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:25.158 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:25.157+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4872 3.2 3:47f58163:::rbd_header.112981949474:head [notify cookie 139868899027616] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.450013+0000", 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.770594752, 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017583306999999999, 2024-09-15T00:09:25.223 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4872 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450013+0000", 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450013+0000", 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450014+0000", 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.32e-07 2024-09-15T00:09:25.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450016+0000", 2024-09-15T00:09:25.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6879999999999999e-06 2024-09-15T00:09:25.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450023+0000", 2024-09-15T00:09:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6120000000000004e-06 2024-09-15T00:09:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450028+0000", 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0459999999999996e-06 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.467407+0000", 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017379296999999998 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.467466+0000", 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8678999999999998e-05 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.467596+0000", 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000130553 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:4109 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.450065+0000", 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7705424929999998, 2024-09-15T00:09:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017749141999999999, 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4109 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450065+0000", 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450065+0000", 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450066+0000", 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4099999999999998e-07 2024-09-15T00:09:25.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450070+0000", 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0119999999999999e-06 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450079+0000", 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5750000000000001e-06 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450085+0000", 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3409999999999999e-06 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.467647+0000", 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017561845999999999 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.467723+0000", 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5785999999999996e-05 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.467815+0000", 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1941000000000006e-05 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4873 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.450144+0000", 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.770463737, 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017906427999999999, 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4873 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450144+0000", 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450144+0000", 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450145+0000", 2024-09-15T00:09:25.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2600000000000002e-07 2024-09-15T00:09:25.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450148+0000", 2024-09-15T00:09:25.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3050000000000001e-06 2024-09-15T00:09:25.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450158+0000", 2024-09-15T00:09:25.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7110000000000007e-06 2024-09-15T00:09:25.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450164+0000", 2024-09-15T00:09:25.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9839999999999998e-06 2024-09-15T00:09:25.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.467865+0000", 2024-09-15T00:09:25.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017701293 2024-09-15T00:09:25.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.467946+0000", 2024-09-15T00:09:25.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0508999999999998e-05 2024-09-15T00:09:25.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468051+0000", 2024-09-15T00:09:25.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0001049 2024-09-15T00:09:25.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.242 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.242 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4876 3.2 3:47f58163:::rbd_header.112981949474:head [notify cookie 139868898659552] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.243 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.450289+0000", 2024-09-15T00:09:25.243 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7703191440000001, 2024-09-15T00:09:25.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017959869999999999, 2024-09-15T00:09:25.243 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.244 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.244 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.244 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4876 2024-09-15T00:09:25.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.244 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450289+0000", 2024-09-15T00:09:25.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450289+0000", 2024-09-15T00:09:25.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450290+0000", 2024-09-15T00:09:25.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0999999999999997e-07 2024-09-15T00:09:25.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450294+0000", 2024-09-15T00:09:25.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9049999999999996e-06 2024-09-15T00:09:25.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450310+0000", 2024-09-15T00:09:25.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5724999999999999e-05 2024-09-15T00:09:25.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450317+0000", 2024-09-15T00:09:25.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9e-06 2024-09-15T00:09:25.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468094+0000", 2024-09-15T00:09:25.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017777384 2024-09-15T00:09:25.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468154+0000", 2024-09-15T00:09:25.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9206000000000002e-05 2024-09-15T00:09:25.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468249+0000", 2024-09-15T00:09:25.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4939999999999996e-05 2024-09-15T00:09:25.252 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.253 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.253 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.253 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4877 3.2 3:47f58163:::rbd_header.112981949474:head [notify cookie 139868899049552] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.253 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.450343+0000", 2024-09-15T00:09:25.254 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7702647989999996, 2024-09-15T00:09:25.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018146171999999999, 2024-09-15T00:09:25.254 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4877 2024-09-15T00:09:25.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.255 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450343+0000", 2024-09-15T00:09:25.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450343+0000", 2024-09-15T00:09:25.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450344+0000", 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2300000000000001e-07 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450347+0000", 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.704e-06 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450356+0000", 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2749999999999995e-06 2024-09-15T00:09:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450362+0000", 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1759999999999998e-06 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468293+0000", 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017931474999999999 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468359+0000", 2024-09-15T00:09:25.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5555000000000005e-05 2024-09-15T00:09:25.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468489+0000", 2024-09-15T00:09:25.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000130364 2024-09-15T00:09:25.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4879 3.2 3:47f58163:::rbd_header.112981949474:head [notify cookie 139868899061680] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.450371+0000", 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7702366779999998, 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018279725, 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4879 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450371+0000", 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450371+0000", 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450372+0000", 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6499999999999999e-07 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450375+0000", 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8229999999999999e-06 2024-09-15T00:09:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450383+0000", 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9900000000000003e-06 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450390+0000", 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1399999999999997e-06 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468525+0000", 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018134962000000001 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468566+0000", 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1153999999999999e-05 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468651+0000", 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5191000000000005e-05 2024-09-15T00:09:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4880 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.450742+0000", 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7698657689999999, 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018135931000000001, 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.266 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4880 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450742+0000", 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450742+0000", 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450743+0000", 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.062e-06 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450749+0000", 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3480000000000003e-06 2024-09-15T00:09:25.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450759+0000", 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0180000000000001e-05 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450765+0000", 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7209999999999997e-06 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468694+0000", 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017928572 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468779+0000", 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4635000000000005e-05 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468878+0000", 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9413000000000005e-05 2024-09-15T00:09:25.270 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:4110 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.450752+0000", 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.769856388, 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018273075999999999, 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4110 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450752+0000", 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450752+0000", 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450753+0000", 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.173e-06 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450758+0000", 2024-09-15T00:09:25.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0300000000000001e-06 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450769+0000", 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1056999999999999e-05 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.450776+0000", 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.542e-06 2024-09-15T00:09:25.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468914+0000", 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018137266999999999 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.468952+0000", 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8547999999999998e-05 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469025+0000", 2024-09-15T00:09:25.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2459000000000005e-05 2024-09-15T00:09:25.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.276 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.277 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4881 3.2 3:47f58163:::rbd_header.112981949474:head [notify cookie 139868898670224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.277 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.451230+0000", 2024-09-15T00:09:25.280 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7693778189999998, 2024-09-15T00:09:25.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018037480000000002, 2024-09-15T00:09:25.280 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.281 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.281 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.281 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4881 2024-09-15T00:09:25.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.282 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451230+0000", 2024-09-15T00:09:25.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451230+0000", 2024-09-15T00:09:25.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451231+0000", 2024-09-15T00:09:25.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7599999999999996e-07 2024-09-15T00:09:25.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451235+0000", 2024-09-15T00:09:25.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2339999999999996e-06 2024-09-15T00:09:25.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451243+0000", 2024-09-15T00:09:25.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3310000000000004e-06 2024-09-15T00:09:25.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451250+0000", 2024-09-15T00:09:25.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3980000000000004e-06 2024-09-15T00:09:25.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469060+0000", 2024-09-15T00:09:25.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017810487999999999 2024-09-15T00:09:25.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469137+0000", 2024-09-15T00:09:25.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6527999999999996e-05 2024-09-15T00:09:25.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469268+0000", 2024-09-15T00:09:25.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013072499999999999 2024-09-15T00:09:25.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.290 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.290 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4882 3.2 3:47f58163:::rbd_header.112981949474:head [notify cookie 139868899082944] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.290 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.451410+0000", 2024-09-15T00:09:25.291 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7691978749999997, 2024-09-15T00:09:25.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018049757999999999, 2024-09-15T00:09:25.291 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.292 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.292 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4882 2024-09-15T00:09:25.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.292 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451410+0000", 2024-09-15T00:09:25.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451410+0000", 2024-09-15T00:09:25.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451411+0000", 2024-09-15T00:09:25.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7999999999999995e-07 2024-09-15T00:09:25.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451415+0000", 2024-09-15T00:09:25.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.792e-06 2024-09-15T00:09:25.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451428+0000", 2024-09-15T00:09:25.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2198000000000001e-05 2024-09-15T00:09:25.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451433+0000", 2024-09-15T00:09:25.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8369999999999996e-06 2024-09-15T00:09:25.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469299+0000", 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017866565000000001 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469356+0000", 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6994999999999999e-05 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469460+0000", 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010369099999999999 2024-09-15T00:09:25.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4883 3.2 3:47f58163:::rbd_header.112981949474:head [notify cookie 139868899095648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.451454+0000", 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7691537630000003, 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018181965000000001, 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.299 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4883 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451454+0000", 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451454+0000", 2024-09-15T00:09:25.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451455+0000", 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5700000000000002e-07 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451459+0000", 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8619999999999996e-06 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451466+0000", 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4749999999999996e-06 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451472+0000", 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6110000000000001e-06 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469505+0000", 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018033449999999999 2024-09-15T00:09:25.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469543+0000", 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7644000000000001e-05 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469636+0000", 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3366e-05 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4884 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.451485+0000", 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7691232179999998, 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018288746000000002, 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4884 2024-09-15T00:09:25.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451485+0000", 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451485+0000", 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451485+0000", 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.44e-07 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451489+0000", 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6660000000000001e-06 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451503+0000", 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.428e-05 2024-09-15T00:09:25.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451510+0000", 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6819999999999997e-06 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469681+0000", 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018170817999999998 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469724+0000", 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3115999999999999e-05 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469773+0000", 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9539999999999997e-05 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:4111 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.308 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.451652+0000", 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.768955729, 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018266261999999998, 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4111 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451652+0000", 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451652+0000", 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451653+0000", 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3200000000000004e-07 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451657+0000", 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3939999999999998e-06 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451667+0000", 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5540000000000003e-06 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451672+0000", 2024-09-15T00:09:25.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8149999999999996e-06 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469789+0000", 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018117352 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469823+0000", 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3762e-05 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469918+0000", 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5552999999999998e-05 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4885 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.451920+0000", 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.768687645, 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018180662, 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4885 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451920+0000", 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451920+0000", 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451921+0000", 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451924+0000", 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8820000000000001e-06 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451935+0000", 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1435e-05 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451939+0000", 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9099999999999998e-06 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.469958+0000", 2024-09-15T00:09:25.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018019086 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470033+0000", 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4791999999999998e-05 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470101+0000", 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8028000000000007e-05 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:4112 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.451935+0000", 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7686724859999998, 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018294823000000002, 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4112 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.318 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451935+0000", 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451935+0000", 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451936+0000", 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2200000000000002e-07 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451940+0000", 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.303e-06 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451946+0000", 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8229999999999997e-06 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.451950+0000", 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9890000000000003e-06 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470129+0000", 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018178778999999999 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470163+0000", 2024-09-15T00:09:25.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4239999999999997e-05 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470230+0000", 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6767000000000006e-05 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4886 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.452371+0000", 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7682371559999996, 2024-09-15T00:09:25.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017968728, 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4886 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452371+0000", 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452371+0000", 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452371+0000", 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6700000000000003e-07 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452375+0000", 2024-09-15T00:09:25.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1889999999999999e-06 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452381+0000", 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3990000000000001e-06 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452385+0000", 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470260+0000", 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017875484000000001 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470294+0000", 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4113999999999999e-05 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470339+0000", 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4994999999999999e-05 2024-09-15T00:09:25.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:4113 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.452373+0000", 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7682344360000002, 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.018118427999999999, 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:25.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4113 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452373+0000", 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452373+0000", 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452374+0000", 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452378+0000", 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3119999999999998e-06 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452384+0000", 2024-09-15T00:09:25.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.793e-06 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.452388+0000", 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8099999999999999e-06 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470375+0000", 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017987067999999998 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470428+0000", 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2219000000000003e-05 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470492+0000", 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4350999999999994e-05 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:4114 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.331 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.453283+0000", 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7673251519999997, 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017476782999999999, 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4114 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453283+0000", 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453283+0000", 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453283+0000", 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1399999999999996e-07 2024-09-15T00:09:25.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453287+0000", 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6559999999999998e-06 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453294+0000", 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8580000000000002e-06 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453298+0000", 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1230000000000002e-06 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470647+0000", 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017348631 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470692+0000", 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4987000000000001e-05 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470760+0000", 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7813999999999998e-05 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:4888 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.453488+0000", 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.767119696, 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017444154, 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4888 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453488+0000", 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453488+0000", 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453489+0000", 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2799999999999996e-07 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453492+0000", 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.791e-06 2024-09-15T00:09:25.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453497+0000", 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.891e-06 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453501+0000", 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4979999999999998e-06 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470804+0000", 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017303392000000001 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470848+0000", 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3411000000000002e-05 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470932+0000", 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4642999999999996e-05 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.340 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:4115 3.2 3:47f58163:::rbd_header.112981949474:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:20.453534+0000", 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7670738779999997, 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017537371, 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:09:25.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4115 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453534+0000", 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453534+0000", 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453535+0000", 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4300000000000003e-07 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453538+0000", 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3680000000000002e-06 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453544+0000", 2024-09-15T00:09:25.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2029999999999997e-06 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.453550+0000", 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.470973+0000", 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.017423174 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.471010+0000", 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6788000000000001e-05 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:20.471071+0000", 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1597e-05 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:25.347 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:25.347 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:25.345+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:25.448 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:25.447+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:25.495 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:25.495 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:25.495 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:25.495 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:25.503 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:25.549 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:25.547+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:25.649 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:25.648+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:25.651 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:25.651 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:25.651 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:25.651 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:25.652 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:25.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:25.751 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:25.750+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1260 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2023424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413708011000001, 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027065815, 2024-09-15T00:09:25.769 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1260 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352960+0000", 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9559999999999997e-06 2024-09-15T00:09:25.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352968+0000", 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5939999999999996e-06 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352973+0000", 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8409999999999999e-06 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379911+0000", 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.026938004000000002 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379958+0000", 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7607e-05 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380018+0000", 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9438999999999998e-05 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1269 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413674858, 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027165930000000001, 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1269 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9100000000000004e-07 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352988+0000", 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4779999999999998e-06 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353002+0000", 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3699999999999999e-05 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353007+0000", 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5310000000000001e-06 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380046+0000", 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027038801000000001 2024-09-15T00:09:25.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380090+0000", 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380151+0000", 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1210000000000005e-05 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1277 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413643796999999, 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027261463999999999, 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1277 2024-09-15T00:09:25.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353026+0000", 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3379999999999994e-06 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353034+0000", 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8790000000000003e-06 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353039+0000", 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4489999999999998e-06 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380180+0000", 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.02714068 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380226+0000", 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5776999999999999e-05 2024-09-15T00:09:25.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380277+0000", 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1922000000000002e-05 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1285 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413603052999999, 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027368669000000002, 2024-09-15T00:09:25.783 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1285 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1399999999999997e-07 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353060+0000", 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4310000000000001e-06 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353067+0000", 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.746e-06 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353073+0000", 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.541e-06 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380305+0000", 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0272325 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380349+0000", 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3995000000000001e-05 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380425+0000", 2024-09-15T00:09:25.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5941999999999999e-05 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1286 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413579137999999, 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027475013999999999, 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.788 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1286 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353087+0000", 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1299999999999998e-06 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353089+0000", 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.587e-06 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353096+0000", 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1409999999999999e-06 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353102+0000", 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380455+0000", 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027352936000000001 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380501+0000", 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6236999999999998e-05 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380556+0000", 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4287999999999999e-05 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1292 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413545931000002, 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027570794999999999, 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1292 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.793 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:25.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4700000000000001e-07 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353117+0000", 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.295e-06 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353130+0000", 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2738999999999999e-05 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353141+0000", 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1885000000000001e-05 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380585+0000", 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027443263999999998 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380628+0000", 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3115999999999999e-05 2024-09-15T00:09:25.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380685+0000", 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6848999999999997e-05 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1294 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413509700999999, 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027661577, 2024-09-15T00:09:25.797 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1294 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353151+0000", 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353153+0000", 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353160+0000", 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353165+0000", 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0340000000000004e-06 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380714+0000", 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027549081 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380758+0000", 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4323000000000002e-05 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380812+0000", 2024-09-15T00:09:25.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3414000000000001e-05 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1314 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413253820000001, 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027532068, 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.802 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1314 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:25.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353407+0000", 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6300000000000004e-07 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353411+0000", 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0099999999999997e-06 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353421+0000", 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0309000000000001e-05 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353428+0000", 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7750000000000004e-06 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380839+0000", 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027410994000000001 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380883+0000", 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4079000000000003e-05 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380938+0000", 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5138e-05 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1320 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:25.806 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413220486, 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027624342999999999, 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1320 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353440+0000", 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0399999999999996e-07 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353442+0000", 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3800000000000001e-06 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353449+0000", 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3250000000000002e-06 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353475+0000", 2024-09-15T00:09:25.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.584e-05 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380968+0000", 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027492249999999999 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381010+0000", 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2228999999999998e-05 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381064+0000", 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3814999999999999e-05 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1321 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.413173633, 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027730890000000001, 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.811 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1321 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:25.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353487+0000", 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6499999999999999e-07 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353489+0000", 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7240000000000001e-06 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353504+0000", 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4636e-05 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353510+0000", 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7060000000000001e-06 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381091+0000", 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027581668 2024-09-15T00:09:25.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381158+0000", 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6229000000000006e-05 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381217+0000", 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9361999999999998e-05 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1324 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.412904658999999, 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027587508, 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1324 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.816 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353756+0000", 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6800000000000009e-07 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353760+0000", 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1099999999999996e-06 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353772+0000", 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1464000000000001e-05 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353779+0000", 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3470000000000001e-06 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381246+0000", 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027467155 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381287+0000", 2024-09-15T00:09:25.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0553999999999998e-05 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381343+0000", 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5909999999999998e-05 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1332 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:25.820 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.412869579999999, 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027695646000000001, 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1332 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353791+0000", 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.58e-07 2024-09-15T00:09:25.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353793+0000", 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3350000000000001e-06 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353800+0000", 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2599999999999999e-06 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353806+0000", 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6470000000000003e-06 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381370+0000", 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027564298000000001 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381429+0000", 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9302999999999998e-05 2024-09-15T00:09:25.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381486+0000", 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6344999999999997e-05 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1333 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.412844280000002, 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027796923000000001, 2024-09-15T00:09:25.825 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1333 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:25.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353816+0000", 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353818+0000", 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.137e-06 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353824+0000", 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3439999999999997e-06 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353829+0000", 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:09:25.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381516+0000", 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027686319000000001 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381558+0000", 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1927e-05 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381612+0000", 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4778999999999998e-05 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1353 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2039808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.412786027999999, 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027865687, 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.830 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1353 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.99e-07 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353879+0000", 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4089999999999996e-06 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353887+0000", 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1470000000000001e-06 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353893+0000", 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6840000000000001e-06 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381640+0000", 2024-09-15T00:09:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027747720999999999 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381683+0000", 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2122000000000001e-05 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381739+0000", 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6904999999999998e-05 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1359 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.412583441999999, 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.02778951, 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1359 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.835 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354077+0000", 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7000000000000003e-07 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354090+0000", 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3158e-05 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354102+0000", 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1572e-05 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354115+0000", 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3297e-05 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381769+0000", 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027653499000000002 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381811+0000", 2024-09-15T00:09:25.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2348000000000002e-05 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381866+0000", 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4765999999999997e-05 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1360 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.412533087, 2024-09-15T00:09:25.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027863138999999999, 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1360 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:25.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354130+0000", 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.542e-06 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354139+0000", 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.85e-06 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354144+0000", 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5550000000000002e-06 2024-09-15T00:09:25.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381895+0000", 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027750654 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381937+0000", 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2664999999999999e-05 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381990+0000", 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1363 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.412494866999999, 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027949321999999999, 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1363 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354166+0000", 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7199999999999998e-07 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354169+0000", 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.207e-06 2024-09-15T00:09:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354178+0000", 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.738e-06 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354184+0000", 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1040000000000003e-06 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382019+0000", 2024-09-15T00:09:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027835069 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382059+0000", 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9988999999999999e-05 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382114+0000", 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5543e-05 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1370 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.412467365000001, 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028036106000000002, 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1370 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354193+0000", 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354195+0000", 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1090000000000001e-06 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354210+0000", 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5423000000000001e-05 2024-09-15T00:09:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354217+0000", 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2269999999999998e-06 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382138+0000", 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027921069999999999 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382177+0000", 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9412999999999997e-05 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382228+0000", 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1437999999999998e-05 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1372 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.412431502, 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028122709999999999, 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1372 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354229+0000", 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8599999999999998e-07 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354232+0000", 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.757e-06 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354239+0000", 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3710000000000002e-06 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354245+0000", 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6300000000000003e-06 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382259+0000", 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028014292999999999 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382301+0000", 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2354999999999997e-05 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382351+0000", 2024-09-15T00:09:25.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9617999999999998e-05 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4574.0:298 3.8 3:13770e18:::rbd_data.112910301a74.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 7764992~33792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:14.691221+0000", 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.075439125999999, 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.045417928000000003, 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:25.858 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4574", 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1245848753", 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 298 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.691221+0000", 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.691221+0000", 2024-09-15T00:09:25.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.691222+0000", 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4079999999999999e-06 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732525+0000", 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.041302740999999997 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732539+0000", 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3821e-05 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732544+0000", 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732595+0000", 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0726000000000002e-05 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732697+0000", 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101612 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 1", 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.733016+0000", 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000319079 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736203+0000", 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031871040000000001 2024-09-15T00:09:25.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736558+0000", 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00035541099999999998 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736593+0000", 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4715999999999998e-05 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736639+0000", 2024-09-15T00:09:25.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5668999999999999e-05 2024-09-15T00:09:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:25.864 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:25.864 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:25.864 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:25.866 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:25.866 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:25.864+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:25.885 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:09:25.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:25.966+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:25.982 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:25.982 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:25.982 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:25.982 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:25.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:26.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:26.031 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:26.048 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:26.064 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:26.065 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:26.068 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:26.066+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:26.081 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:26.081 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:26.096 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:26.096 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:26.099 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:26.099 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:26.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:26.116 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:26.116 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:26.189 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:26.187+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:26.247 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:26.247 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:26.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:26.299 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:26.295+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:26.371 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:26.369+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:26.399 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:26.399 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:26.407 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:26.471 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:26.470+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:26.550 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:26.550 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:26.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:26.573 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:26.572+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:26.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:26.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:26.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:26.714 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:26.712+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:26.771 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:26.772 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:26.774 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:26.772+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:26.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:26.874 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:26.873+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:26.879 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:09:26.946 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:26.946 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:26.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:26.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:26.974+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:27.076 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:27.075+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:27.120 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:27.120 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:27.129 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:27.177 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:27.176+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:27.203 INFO:teuthology.orchestra.run.smithi092.stdout:1506 punch from 0x3498c00 to 0x34a3800, (0xac00 bytes) 2024-09-15T00:09:27.210 INFO:teuthology.orchestra.run.smithi092.stdout:1507 punch from 0x602c00 to 0x60c800, (0x9c00 bytes) 2024-09-15T00:09:27.227 INFO:teuthology.orchestra.run.smithi092.stdout:1510 write 0x221a600 thru 0x22231bb (0x8bbc bytes) 2024-09-15T00:09:27.227 INFO:teuthology.orchestra.run.smithi092.stdout:1513 read 0x829a600 thru 0x82a4d0f (0xa710 bytes) 2024-09-15T00:09:27.227 INFO:teuthology.orchestra.run.smithi092.stdout:1515 read 0x201d000 thru 0x2026efa (0x9efb bytes) 2024-09-15T00:09:27.227 INFO:teuthology.orchestra.run.smithi092.stdout:1516 punch from 0x1bf4600 to 0x1bf8800, (0x4200 bytes) 2024-09-15T00:09:27.235 INFO:teuthology.orchestra.run.smithi092.stdout:1517 write 0x93d3c00 thru 0x93e2746 (0xeb47 bytes) 2024-09-15T00:09:27.241 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:27.242 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:27.251 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:27.253 INFO:teuthology.orchestra.run.smithi092.stdout:1523 write 0xcd67e00 thru 0xcd77c3c (0xfe3d bytes) 2024-09-15T00:09:27.257 INFO:teuthology.orchestra.run.smithi092.stdout:1524 trunc from 0xcd77e00 to 0x9ce6e00 2024-09-15T00:09:27.276 INFO:teuthology.orchestra.run.smithi092.stdout:1525 write 0x9bd6000 thru 0x9bd9327 (0x3328 bytes) 2024-09-15T00:09:27.278 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:27.276+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:27.284 INFO:teuthology.orchestra.run.smithi092.stdout:1526 read 0x7d55400 thru 0x7d5db1f (0x8720 bytes) 2024-09-15T00:09:27.284 INFO:teuthology.orchestra.run.smithi092.stdout:1527 write 0x5db0c00 thru 0x5dbad94 (0xa195 bytes) 2024-09-15T00:09:27.284 INFO:teuthology.orchestra.run.smithi092.stdout:1529 read 0x84b3800 thru 0x84bdbd1 (0xa3d2 bytes) 2024-09-15T00:09:27.285 INFO:teuthology.orchestra.run.smithi092.stdout:1530 read 0x6688e00 thru 0x668f09e (0x629f bytes) 2024-09-15T00:09:27.285 INFO:teuthology.orchestra.run.smithi092.stdout:1532 punch from 0xa0b200 to 0xa19000, (0xde00 bytes) 2024-09-15T00:09:27.285 INFO:teuthology.orchestra.run.smithi092.stdout:1533 clone 13 order 21 su 16384 sc 3 2024-09-15T00:09:27.363 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:27.364 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:27.372 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:27.379 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:27.377+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:27.479 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:27.479 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:27.479 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:27.478+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:27.488 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:27.580 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:27.579+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:27.595 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:27.595 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:27.603 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:27.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:27.679+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:27.710 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:27.710 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:27.718 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:27.803 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:27.801+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:27.867 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:27.867 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:27.876 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:27.917 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:27.915+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:27.968 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone12 from 0x9ce6e00 (overlap 0x9ce6e00) to 0x2f12000 2024-09-15T00:09:27.983 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:27.981+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:28.030 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:28.031 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:28.039 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:28.123 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:28.121+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:28.127 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:28.185 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:28.183+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:28.192 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:28.192 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:28.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:28.261 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:28.261 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:28.286 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:28.284+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:28.294 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:28.294 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:28.311 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:28.312 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:28.319 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-15T00:09:28.328 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:28.328 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:28.330 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:28.330 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:28.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:28.330 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:28.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.330 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1260 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2023424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.330 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974732147000001, 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027065815, 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1260 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352952+0000", 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352960+0000", 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9559999999999997e-06 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352968+0000", 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5939999999999996e-06 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352973+0000", 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8409999999999999e-06 2024-09-15T00:09:28.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379911+0000", 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.026938004000000002 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.379958+0000", 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7607e-05 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380018+0000", 2024-09-15T00:09:28.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9438999999999998e-05 2024-09-15T00:09:28.335 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.335 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.335 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.335 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1269 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.335 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:28.335 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974698994000001, 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027165930000000001, 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1269 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352985+0000", 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9100000000000004e-07 2024-09-15T00:09:28.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.352988+0000", 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4779999999999998e-06 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353002+0000", 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3699999999999999e-05 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353007+0000", 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5310000000000001e-06 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380046+0000", 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027038801000000001 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380090+0000", 2024-09-15T00:09:28.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380151+0000", 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1210000000000005e-05 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1277 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:28.340 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974667932999999, 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027261463999999999, 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1277 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353016+0000", 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1899999999999998e-07 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353026+0000", 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3379999999999994e-06 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353034+0000", 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8790000000000003e-06 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353039+0000", 2024-09-15T00:09:28.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4489999999999998e-06 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380180+0000", 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.02714068 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380226+0000", 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5776999999999999e-05 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380277+0000", 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1922000000000002e-05 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1285 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974627189, 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027368669000000002, 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1285 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353057+0000", 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1399999999999997e-07 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353060+0000", 2024-09-15T00:09:28.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4310000000000001e-06 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353067+0000", 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.746e-06 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353073+0000", 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.541e-06 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380305+0000", 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0272325 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380349+0000", 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3995000000000001e-05 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380425+0000", 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5941999999999999e-05 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1286 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974603274, 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027475013999999999, 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1286 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353081+0000", 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353087+0000", 2024-09-15T00:09:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1299999999999998e-06 2024-09-15T00:09:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353089+0000", 2024-09-15T00:09:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.587e-06 2024-09-15T00:09:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353096+0000", 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1409999999999999e-06 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353102+0000", 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6949999999999997e-06 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380455+0000", 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027352936000000001 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380501+0000", 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6236999999999998e-05 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380556+0000", 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4287999999999999e-05 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.354 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1292 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974570066999998, 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027570794999999999, 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1292 2024-09-15T00:09:28.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353114+0000", 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4700000000000001e-07 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353117+0000", 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.295e-06 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353130+0000", 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2738999999999999e-05 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353141+0000", 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1885000000000001e-05 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380585+0000", 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027443263999999998 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380628+0000", 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3115999999999999e-05 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380685+0000", 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6848999999999997e-05 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1294 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974533836999999, 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027661577, 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1294 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353150+0000", 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353151+0000", 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353153+0000", 2024-09-15T00:09:28.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353160+0000", 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353165+0000", 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0340000000000004e-06 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380714+0000", 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027549081 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380758+0000", 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4323000000000002e-05 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380812+0000", 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3414000000000001e-05 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.363 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1314 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2031616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974277956000002, 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027532068, 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.364 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1314 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353406+0000", 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353407+0000", 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6300000000000004e-07 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353411+0000", 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0099999999999997e-06 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353421+0000", 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0309000000000001e-05 2024-09-15T00:09:28.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353428+0000", 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7750000000000004e-06 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380839+0000", 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027410994000000001 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380883+0000", 2024-09-15T00:09:28.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4079000000000003e-05 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380938+0000", 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5138e-05 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1320 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:28.368 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974244622000001, 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027624342999999999, 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1320 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:28.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353439+0000", 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353440+0000", 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0399999999999996e-07 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353442+0000", 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3800000000000001e-06 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353449+0000", 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3250000000000002e-06 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353475+0000", 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.584e-05 2024-09-15T00:09:28.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.380968+0000", 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027492249999999999 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381010+0000", 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2228999999999998e-05 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381064+0000", 2024-09-15T00:09:28.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3814999999999999e-05 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1321 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.974197769, 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027730890000000001, 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.373 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1321 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353486+0000", 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353487+0000", 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6499999999999999e-07 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353489+0000", 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7240000000000001e-06 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353504+0000", 2024-09-15T00:09:28.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4636e-05 2024-09-15T00:09:28.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353510+0000", 2024-09-15T00:09:28.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7060000000000001e-06 2024-09-15T00:09:28.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381091+0000", 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027581668 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381158+0000", 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6229000000000006e-05 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381217+0000", 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9361999999999998e-05 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1324 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.973928794999999, 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027587508, 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1324 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353755+0000", 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353756+0000", 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6800000000000009e-07 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353760+0000", 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1099999999999996e-06 2024-09-15T00:09:28.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353772+0000", 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1464000000000001e-05 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353779+0000", 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3470000000000001e-06 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381246+0000", 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027467155 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381287+0000", 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0553999999999998e-05 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381343+0000", 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5909999999999998e-05 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1332 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.973893715999999, 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027695646000000001, 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.383 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1332 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353790+0000", 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353791+0000", 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.58e-07 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353793+0000", 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3350000000000001e-06 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353800+0000", 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2599999999999999e-06 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353806+0000", 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6470000000000003e-06 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381370+0000", 2024-09-15T00:09:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027564298000000001 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381429+0000", 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9302999999999998e-05 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381486+0000", 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6344999999999997e-05 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1333 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.973868415999998, 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027796923000000001, 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1333 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353815+0000", 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353816+0000", 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353818+0000", 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.137e-06 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353824+0000", 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3439999999999997e-06 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353829+0000", 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9250000000000003e-06 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381516+0000", 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027686319000000001 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381558+0000", 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1927e-05 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381612+0000", 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4778999999999998e-05 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1353 3.18 3:184f6d86:::rbd_data.1129feb48f4b.000000000000000a:4 [read 2039808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.973810164, 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027865687, 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1353 2024-09-15T00:09:28.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353874+0000", 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.99e-07 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353879+0000", 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4089999999999996e-06 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353887+0000", 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1470000000000001e-06 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.353893+0000", 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6840000000000001e-06 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381640+0000", 2024-09-15T00:09:28.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027747720999999999 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381683+0000", 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2122000000000001e-05 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381739+0000", 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6904999999999998e-05 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1359 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.973607577999999, 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.02778951, 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1359 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354076+0000", 2024-09-15T00:09:28.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354077+0000", 2024-09-15T00:09:28.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7000000000000003e-07 2024-09-15T00:09:28.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354090+0000", 2024-09-15T00:09:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3158e-05 2024-09-15T00:09:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354102+0000", 2024-09-15T00:09:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1572e-05 2024-09-15T00:09:28.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354115+0000", 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3297e-05 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381769+0000", 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027653499000000002 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381811+0000", 2024-09-15T00:09:28.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2348000000000002e-05 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381866+0000", 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4765999999999997e-05 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1360 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.973557223, 2024-09-15T00:09:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027863138999999999, 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1360 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354127+0000", 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3099999999999998e-07 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354130+0000", 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.542e-06 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354139+0000", 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.85e-06 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354144+0000", 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5550000000000002e-06 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381895+0000", 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027750654 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381937+0000", 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2664999999999999e-05 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.381990+0000", 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.407 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1363 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.973519003, 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027949321999999999, 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.408 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1363 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354165+0000", 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354166+0000", 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7199999999999998e-07 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354169+0000", 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.207e-06 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354178+0000", 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.738e-06 2024-09-15T00:09:28.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354184+0000", 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1040000000000003e-06 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382019+0000", 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027835069 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382059+0000", 2024-09-15T00:09:28.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9988999999999999e-05 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382114+0000", 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5543e-05 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1370 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.973491501000002, 2024-09-15T00:09:28.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028036106000000002, 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1370 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:28.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354192+0000", 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354193+0000", 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2599999999999998e-07 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354195+0000", 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1090000000000001e-06 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354210+0000", 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5423000000000001e-05 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354217+0000", 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2269999999999998e-06 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382138+0000", 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.027921069999999999 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382177+0000", 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9412999999999997e-05 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382228+0000", 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1437999999999998e-05 2024-09-15T00:09:28.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4507.0:1372 3.18 3:1def372c:::rbd_data.1129feb48f4b.0000000000000012:4 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.973455638000001, 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028122709999999999, 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4507", 2024-09-15T00:09:28.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1188719296", 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1372 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354228+0000", 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354229+0000", 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8599999999999998e-07 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354232+0000", 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.757e-06 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354239+0000", 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3710000000000002e-06 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.354245+0000", 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6300000000000003e-06 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382259+0000", 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.028014292999999999 2024-09-15T00:09:28.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382301+0000", 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2354999999999997e-05 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:07.382351+0000", 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9617999999999998e-05 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4574.0:298 3.8 3:13770e18:::rbd_data.112910301a74.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 7764992~33792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:14.691221+0000", 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 13.636463261999999, 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.045417928000000003, 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4574", 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1245848753", 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 298 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.422 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.691221+0000", 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.691221+0000", 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.691222+0000", 2024-09-15T00:09:28.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4079999999999999e-06 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732525+0000", 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.041302740999999997 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732539+0000", 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3821e-05 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732544+0000", 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732595+0000", 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0726000000000002e-05 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.732697+0000", 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101612 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 1", 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.733016+0000", 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000319079 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736203+0000", 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031871040000000001 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736558+0000", 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00035541099999999998 2024-09-15T00:09:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736593+0000", 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4715999999999998e-05 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:14.736639+0000", 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5668999999999999e-05 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:28.428 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:28.428 INFO:teuthology.orchestra.run.smithi092.stdout:1535 punch from 0xf4a400 to 0xf57200, (0xce00 bytes) 2024-09-15T00:09:28.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:28.430 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:28.430 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:28.430 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:28.431 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:28.431 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:28.427+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:28.440 INFO:teuthology.orchestra.run.smithi092.stdout:1536 read 0x73e9600 thru 0x73f789e (0xe29f bytes) 2024-09-15T00:09:28.441 INFO:teuthology.orchestra.run.smithi092.stdout:1538 write 0x1f20200 thru 0x1f269e3 (0x67e4 bytes) 2024-09-15T00:09:28.442 INFO:teuthology.orchestra.run.smithi092.stdout:1540 trunc from 0x9ce6e00 to 0xa2a0400 2024-09-15T00:09:28.491 INFO:teuthology.orchestra.run.smithi092.stdout:1541 trunc from 0xa2a0400 to 0xc61e000 2024-09-15T00:09:28.499 INFO:teuthology.orchestra.run.smithi092.stdout:1542 punch from 0x9c5800 to 0x9c7600, (0x1e00 bytes) 2024-09-15T00:09:28.514 INFO:teuthology.orchestra.run.smithi092.stdout:1543 trunc from 0xc61e000 to 0x247f000 2024-09-15T00:09:28.530 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:28.528+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:28.549 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:28.549 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:28.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:28.563 INFO:teuthology.orchestra.run.smithi092.stdout:1548 read 0x2215a00 thru 0x22249df (0xefe0 bytes) 2024-09-15T00:09:28.587 INFO:teuthology.orchestra.run.smithi092.stdout:1549 write 0xda25400 thru 0xda32452 (0xd053 bytes) 2024-09-15T00:09:28.598 INFO:teuthology.orchestra.run.smithi092.stdout:1550 trunc from 0xda32600 to 0x551aa00 2024-09-15T00:09:28.626 INFO:teuthology.orchestra.run.smithi092.stdout:1551 write 0x74ec800 thru 0x74f3c6c (0x746d bytes) 2024-09-15T00:09:28.626 INFO:teuthology.orchestra.run.smithi092.stdout:1553 read 0x16a7c00 thru 0x16b0bb0 (0x8fb1 bytes) 2024-09-15T00:09:28.631 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:28.629+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:28.631 INFO:teuthology.orchestra.run.smithi092.stdout:1554 trunc from 0x74f3e00 to 0xae92e00 2024-09-15T00:09:28.643 INFO:teuthology.orchestra.run.smithi092.stdout:1555 read 0x7a7d400 thru 0x7a89e83 (0xca84 bytes) 2024-09-15T00:09:28.644 INFO:teuthology.orchestra.run.smithi092.stdout:1556 punch from 0x2cbee00 to 0x2cc3800, (0x4a00 bytes) 2024-09-15T00:09:28.645 INFO:teuthology.orchestra.run.smithi092.stdout:1557 punch from 0xa3cd000 to 0xa3d9e00, (0xce00 bytes) 2024-09-15T00:09:28.645 INFO:teuthology.orchestra.run.smithi092.stdout:1558 read 0xa46e800 thru 0xa47e502 (0xfd03 bytes) 2024-09-15T00:09:28.646 INFO:teuthology.orchestra.run.smithi092.stdout:1560 read 0x68ce000 thru 0x68db5f1 (0xd5f2 bytes) 2024-09-15T00:09:28.646 INFO:teuthology.orchestra.run.smithi092.stdout:1562 punch from 0x8cf5200 to 0x8d01800, (0xc600 bytes) 2024-09-15T00:09:28.646 INFO:teuthology.orchestra.run.smithi092.stdout:1563 read 0x2c89e00 thru 0x2c958a1 (0xbaa2 bytes) 2024-09-15T00:09:28.646 INFO:teuthology.orchestra.run.smithi092.stdout:1565 write 0x1a79600 thru 0x1a7e91e (0x531f bytes) 2024-09-15T00:09:28.647 INFO:teuthology.orchestra.run.smithi092.stdout:1567 write 0x5ab1c00 thru 0x5abec37 (0xd038 bytes) 2024-09-15T00:09:28.705 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:28.706 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:28.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:28.732 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:28.730+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:28.817 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:28.817 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:28.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4070 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4640299100000007, 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02702953, 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4070 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350999+0000", 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351003+0000", 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351007+0000", 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6820000000000001e-06 2024-09-15T00:09:28.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351010+0000", 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7269999999999999e-06 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377929+0000", 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026919887999999999 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377969+0000", 2024-09-15T00:09:28.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9909e-05 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378028+0000", 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8578000000000001e-05 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4071 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:28.822 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4640150300000006, 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027136008, 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4071 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:28.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351014+0000", 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351015+0000", 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7039999999999999e-06 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351020+0000", 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351023+0000", 2024-09-15T00:09:28.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378053+0000", 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027029610999999999 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378094+0000", 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1826999999999998e-05 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378149+0000", 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4815000000000003e-05 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4072 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4639903509999996, 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027223404999999999, 2024-09-15T00:09:28.827 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4072 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-15T00:09:28.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351040+0000", 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351046+0000", 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351051+0000", 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378171+0000", 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027120590999999999 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378209+0000", 2024-09-15T00:09:28.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7308999999999998e-05 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378261+0000", 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2753999999999997e-05 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4074 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:28.832 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4639714559999994, 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027334509999999999, 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4074 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9200000000000002e-07 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351059+0000", 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.012e-06 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351065+0000", 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2129999999999996e-06 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351070+0000", 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8470000000000003e-06 2024-09-15T00:09:28.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378285+0000", 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027214449000000002 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378334+0000", 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378391+0000", 2024-09-15T00:09:28.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7046000000000002e-05 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4075 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4639525849999995, 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027546923000000001, 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4075 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:28.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351078+0000", 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.905e-06 2024-09-15T00:09:28.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351085+0000", 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351090+0000", 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378423+0000", 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027333051000000001 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378523+0000", 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9959999999999998e-05 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378623+0000", 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9962000000000003e-05 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4077 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4639299010000002, 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027660619000000001, 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4077 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351099+0000", 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351101+0000", 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351107+0000", 2024-09-15T00:09:28.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7980000000000002e-06 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351111+0000", 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378657+0000", 2024-09-15T00:09:28.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027545463999999999 2024-09-15T00:09:28.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378702+0000", 2024-09-15T00:09:28.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5809999999999997e-05 2024-09-15T00:09:28.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378759+0000", 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4078 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4639019009999998, 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027808405000000001, 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4078 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351127+0000", 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351129+0000", 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-15T00:09:28.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351135+0000", 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3820000000000001e-06 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351141+0000", 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2259999999999996e-06 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378785+0000", 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027644592999999999 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378855+0000", 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0276000000000003e-05 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378935+0000", 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9361000000000001e-05 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4079 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4638802640000002, 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027913778, 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4079 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:28.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351150+0000", 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9010000000000001e-06 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351157+0000", 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351162+0000", 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378958+0000", 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027795786999999999 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379004+0000", 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6124000000000001e-05 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379062+0000", 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7630999999999998e-05 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4081 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4638565830000001, 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016658, 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4081 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.857 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:28.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351174+0000", 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351181+0000", 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9070000000000001e-06 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351186+0000", 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.651e-06 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379085+0000", 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027898757999999999 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379130+0000", 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5126000000000001e-05 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379188+0000", 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8548999999999998e-05 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4082 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4638365289999999, 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028118457999999999, 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4082 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351194+0000", 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7519999999999999e-06 2024-09-15T00:09:28.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351200+0000", 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.801e-06 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351205+0000", 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379213+0000", 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028007539000000001 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379254+0000", 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1817999999999998e-05 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379310+0000", 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5850000000000002e-05 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4083 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4638148090000005, 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028218838999999999, 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.866 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4083 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351214+0000", 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351216+0000", 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351222+0000", 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-15T00:09:28.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351226+0000", 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379333+0000", 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028106740000000002 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379376+0000", 2024-09-15T00:09:28.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3581000000000002e-05 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379432+0000", 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5868000000000001e-05 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4085 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:28.870 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4637863049999993, 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311823, 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4085 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0699999999999998e-07 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351244+0000", 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.782e-06 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351248+0000", 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6289999999999999e-06 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351250+0000", 2024-09-15T00:09:28.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379457+0000", 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028206479 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379499+0000", 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1637000000000001e-05 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379554+0000", 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5248999999999999e-05 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4086 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4637744640000001, 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02842368, 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4086 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351259+0000", 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9520000000000002e-06 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351262+0000", 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351265+0000", 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379576+0000", 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311228000000001 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379619+0000", 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2595000000000001e-05 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379678+0000", 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4087 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4637601090000008, 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523903999999999, 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4087 2024-09-15T00:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351269+0000", 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351270+0000", 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1090000000000001e-06 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351273+0000", 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9189999999999999e-06 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351275+0000", 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379701+0000", 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028425813000000001 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379742+0000", 2024-09-15T00:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0769000000000002e-05 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379792+0000", 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4088 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4637439749999999, 2024-09-15T00:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028621284, 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4088 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351285+0000", 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351286+0000", 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.186e-06 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351289+0000", 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351292+0000", 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379815+0000", 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523211 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379857+0000", 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1807999999999997e-05 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379906+0000", 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8594999999999998e-05 2024-09-15T00:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4090 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4637328899999993, 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028748587999999999, 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4090 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351296+0000", 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351297+0000", 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1060000000000001e-06 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351300+0000", 2024-09-15T00:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9110000000000002e-06 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351302+0000", 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379947+0000", 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028645177000000001 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379991+0000", 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3680999999999997e-05 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380044+0000", 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2957000000000001e-05 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4091 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4637233349999992, 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028864198000000001, 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4091 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351306+0000", 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351309+0000", 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351312+0000", 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380069+0000", 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028757933999999999 2024-09-15T00:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380113+0000", 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3810000000000002e-05 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380169+0000", 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5887999999999998e-05 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4092 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4637139769999994, 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028973819000000001, 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4092 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351315+0000", 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9999999999999999e-07 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351316+0000", 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.95e-07 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351318+0000", 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8629999999999999e-06 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351321+0000", 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380195+0000", 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028873868 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380238+0000", 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2883999999999998e-05 2024-09-15T00:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380288+0000", 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0096000000000003e-05 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4094 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4637043579999993, 2024-09-15T00:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02907964, 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4094 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351325+0000", 2024-09-15T00:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.054e-06 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351328+0000", 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351331+0000", 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4679999999999999e-06 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380312+0000", 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028981587999999999 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380355+0000", 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2688000000000002e-05 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380404+0000", 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8767999999999998e-05 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4095 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.4636949710000007, 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029185068000000002, 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4095 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351334+0000", 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.04e-07 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351335+0000", 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0300000000000001e-06 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351339+0000", 2024-09-15T00:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351344+0000", 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0560000000000004e-06 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380428+0000", 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02908444 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380470+0000", 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1338000000000003e-05 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380518+0000", 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8724999999999999e-05 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:28.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:09:28.915 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:28.912+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:28.915 INFO:teuthology.orchestra.run.smithi092.stdout:1569 write 0xbcc2e00 thru 0xbcc4d6d (0x1f6e bytes) 2024-09-15T00:09:28.915 INFO:teuthology.orchestra.run.smithi092.stdout:1570 read 0x64ce000 thru 0x64d8a08 (0xaa09 bytes) 2024-09-15T00:09:28.915 INFO:teuthology.orchestra.run.smithi092.stdout:1573 read 0x111a200 thru 0x1125e16 (0xbc17 bytes) 2024-09-15T00:09:28.915 INFO:teuthology.orchestra.run.smithi092.stdout:1574 clone 14 order 25 su 16777216 sc 2 2024-09-15T00:09:28.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:29.015 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:29.013+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:29.019 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:29.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:29.020 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:29.020 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:29.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:29.116 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:29.115+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:29.171 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:29.171 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:29.171 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:29.171 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:29.171 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:29.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:29.221 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:29.219+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:29.318 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:29.317+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:29.324 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3427 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:29.324 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.034375786, 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027724631, 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3427 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:29.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287582+0000", 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2599999999999998e-07 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287584+0000", 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8759999999999999e-06 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287589+0000", 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287594+0000", 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.357e-06 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315217+0000", 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027622785 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315257+0000", 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0228000000000002e-05 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315306+0000", 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9228999999999999e-05 2024-09-15T00:09:29.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3428 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0343577810000006, 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027928562000000001, 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3428 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287600+0000", 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287602+0000", 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287608+0000", 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:09:29.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287613+0000", 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8740000000000003e-06 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315346+0000", 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027733099000000001 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315431+0000", 2024-09-15T00:09:29.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4350000000000004e-05 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315528+0000", 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7374000000000007e-05 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3429 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:29.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0343384709999999, 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028028630999999998, 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3429 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287621+0000", 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287627+0000", 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8440000000000003e-06 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287631+0000", 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5009999999999998e-06 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315556+0000", 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027925241999999999 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315595+0000", 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8512e-05 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315647+0000", 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2318000000000003e-05 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3430 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0343203970000001, 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028122114, 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3430 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287644+0000", 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4219999999999997e-06 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287650+0000", 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0920000000000003e-06 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287654+0000", 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4279999999999998e-06 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315671+0000", 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016592 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315709+0000", 2024-09-15T00:09:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8231e-05 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315759+0000", 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9926999999999998e-05 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3431 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0342968569999993, 2024-09-15T00:09:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02831842, 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3431 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287661+0000", 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287663+0000", 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9570000000000001e-06 2024-09-15T00:09:29.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287668+0000", 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287673+0000", 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315799+0000", 2024-09-15T00:09:29.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028125741999999999 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315891+0000", 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2701000000000006e-05 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315979+0000", 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7287000000000006e-05 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3432 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0342784730000005, 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028430624000000002, 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3432 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287681+0000", 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8530000000000001e-06 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287686+0000", 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4659999999999998e-06 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287691+0000", 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2230000000000001e-06 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316006+0000", 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028315344999999999 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316054+0000", 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8004999999999999e-05 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316109+0000", 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5374999999999998e-05 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3433 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0342607170000004, 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028528593000000001, 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3433 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.351 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287697+0000", 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287699+0000", 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8509999999999999e-06 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287704+0000", 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287710+0000", 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316124+0000", 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028414466999999999 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316171+0000", 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6783999999999998e-05 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316225+0000", 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3891999999999998e-05 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3434 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0342413320000006, 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028632801999999999, 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.355 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3434 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-07 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287718+0000", 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:29.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287724+0000", 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8390000000000003e-06 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287729+0000", 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8389999999999997e-06 2024-09-15T00:09:29.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316250+0000", 2024-09-15T00:09:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028521135 2024-09-15T00:09:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316295+0000", 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4540999999999999e-05 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316349+0000", 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4083999999999999e-05 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3435 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0342106560000008, 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028723243999999998, 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3435 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287747+0000", 2024-09-15T00:09:29.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4900000000000002e-07 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287748+0000", 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4190000000000001e-06 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287752+0000", 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7900000000000001e-06 2024-09-15T00:09:29.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287756+0000", 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1609999999999996e-06 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316375+0000", 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028618932 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316417+0000", 2024-09-15T00:09:29.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316470+0000", 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.365 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3436 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0341956079999992, 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028827854, 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3436 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:29.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287766+0000", 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287771+0000", 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5140000000000001e-06 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287776+0000", 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316494+0000", 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028718144000000001 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316538+0000", 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4129999999999999e-05 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316589+0000", 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1548999999999997e-05 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3437 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0341777160000003, 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028972188999999999, 2024-09-15T00:09:29.370 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3437 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287780+0000", 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287782+0000", 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8419999999999999e-06 2024-09-15T00:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287787+0000", 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3059999999999997e-06 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287791+0000", 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9920000000000001e-06 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316615+0000", 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028824487999999999 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316668+0000", 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2883999999999997e-05 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316752+0000", 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3308999999999998e-05 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3438 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0341606740000007, 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029124331999999999, 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3438 2024-09-15T00:09:29.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287797+0000", 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287799+0000", 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287804+0000", 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7069999999999997e-06 2024-09-15T00:09:29.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287808+0000", 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0720000000000001e-06 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316799+0000", 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028990241999999999 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316845+0000", 2024-09-15T00:09:29.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5837000000000002e-05 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316921+0000", 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6255999999999996e-05 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3439 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0341431350000008, 2024-09-15T00:09:29.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029290665, 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3439 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:29.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287816+0000", 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6980000000000001e-06 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287824+0000", 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2500000000000006e-06 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287829+0000", 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:09:29.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316967+0000", 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029138319999999999 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317047+0000", 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0081999999999997e-05 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317105+0000", 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7497999999999997e-05 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3440 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0341220759999992, 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029395451999999999, 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3440 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:29.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287837+0000", 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287843+0000", 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6799999999999998e-06 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287848+0000", 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317131+0000", 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029283337999999999 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317176+0000", 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5176999999999998e-05 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317231+0000", 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4120999999999999e-05 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3441 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.388 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0341040120000002, 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029496321999999998, 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3441 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287855+0000", 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7099999999999999e-06 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287861+0000", 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287866+0000", 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:29.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317252+0000", 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029386051999999999 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317297+0000", 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5164999999999999e-05 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317349+0000", 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2534000000000002e-05 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3442 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0340847459999996, 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029594697999999999, 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3442 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287873+0000", 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287879+0000", 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287885+0000", 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.755e-06 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287890+0000", 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317375+0000", 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029485457 2024-09-15T00:09:29.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317416+0000", 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1217999999999997e-05 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317467+0000", 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0819999999999998e-05 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3443 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0340614069999994, 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029684676, 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3443 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:29.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287898+0000", 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287904+0000", 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8860000000000001e-06 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287908+0000", 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5650000000000003e-06 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317488+0000", 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029579437 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317530+0000", 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1900999999999997e-05 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317580+0000", 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0603000000000003e-05 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.402 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3444 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.034036189, 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029772823, 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3444 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:09:29.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287931+0000", 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0008e-05 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287938+0000", 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287943+0000", 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317603+0000", 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029659913999999999 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317646+0000", 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2367999999999998e-05 2024-09-15T00:09:29.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317694+0000", 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8201999999999997e-05 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3445 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0340068509999991, 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029863565000000002, 2024-09-15T00:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3445 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287951+0000", 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287953+0000", 2024-09-15T00:09:29.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.765e-06 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287958+0000", 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999996e-06 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287963+0000", 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.634e-06 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317714+0000", 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029751353000000001 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317755+0000", 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0342000000000001e-05 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317814+0000", 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9188999999999999e-05 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.411 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3446 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0339885530000004, 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029967884, 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:29.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3446 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287971+0000", 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287977+0000", 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4330000000000003e-06 2024-09-15T00:09:29.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287981+0000", 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3989999999999997e-06 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317837+0000", 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029855942 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317878+0000", 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467000000000001e-05 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317936+0000", 2024-09-15T00:09:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8100999999999998e-05 2024-09-15T00:09:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.416 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:29.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:29.420 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:29.418+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:29.521 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:29.519+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:29.561 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:29.562 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:29.562 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:29.562 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:29.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:29.656 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:29.655+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:29.716 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:29.716 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:29.722 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:29.720+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:29.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:29.823 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:29.821+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:29.871 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:29.871 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1142 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.986335434000001, 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045190363999999997, 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1142 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882756+0000", 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882773+0000", 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6872999999999999e-05 2024-09-15T00:09:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882780+0000", 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4240000000000004e-06 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882787+0000", 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8510000000000001e-06 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927799+0000", 2024-09-15T00:09:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045011974000000003 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927843+0000", 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4270999999999997e-05 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927946+0000", 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102581 2024-09-15T00:09:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.986286742000001, 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:09:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.986248112, 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:09:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:09:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:09:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:09:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.986222175, 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:09:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.986206106999999, 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:09:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:09:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.986065011999999, 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:09:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:09:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:09:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.985293257, 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.985208225999999, 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.985173209999999, 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:09:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.985147607, 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:09:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.984973341, 2024-09-15T00:09:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:09:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.984742202, 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:29.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:09:29.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1157 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.983960764000001, 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045184281, 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.931 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1157 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885132+0000", 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885139+0000", 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0350000000000004e-06 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885153+0000", 2024-09-15T00:09:29.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3515e-05 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930177+0000", 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045024495999999997 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930249+0000", 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2072999999999999e-05 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930314+0000", 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4826999999999994e-05 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.983930258000001, 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:09:29.936 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:09:29.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:09:29.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:09:29.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:09:29.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:09:29.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.983850518000001, 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.942 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:09:29.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:09:29.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:09:29.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:09:29.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:09:29.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.983645493000001, 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.947 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:09:29.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:09:29.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:09:29.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:29.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.952 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.983433959999999, 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:09:29.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:29.957 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.980877953, 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:09:29.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.978894846999999, 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.962 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.978875276, 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:29.967 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:09:29.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:09:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:09:29.972 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.972 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.972 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.972 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:29.972 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:29.972 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:29.972 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-15T00:09:29.972 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:09:29.972 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T00:09:29.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:09:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:09:29.975 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:29.972+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:29.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:29.987 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone13 2024-09-15T00:09:30.075 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:30.073+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:30.166 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:30.166 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:30.166 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:30.166 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:30.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:30.176 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:30.174+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:30.277 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:30.275+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:30.328 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:30.328 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:30.328 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:30.328 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:30.328 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:30.337 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:30.358 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:09:30.358 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:30.358 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 46, 2024-09-15T00:09:30.358 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "4f532d31-28c1-4236-a74e-6392960baf2b", 2024-09-15T00:09:30.359 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2024-09-15T00:08:32.007191+0000", 2024-09-15T00:09:30.359 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2024-09-15T00:09:29.888994+0000", 2024-09-15T00:09:30.359 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2024-09-15T00:08:42.794810+0000", 2024-09-15T00:09:30.359 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2024-09-15T00:08:33.890963+0000", 2024-09-15T00:09:30.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:09:30.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5804032, 2024-09-15T00:09:30.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2024-09-15T00:09:30.359 INFO:teuthology.orchestra.run.smithi027.stdout: "nodeep-scrub", 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 3, 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 6, 2024-09-15T00:09:30.360 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "squid", 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:42.896076+0000", 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:30.361 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2024-09-15T00:09:30.362 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "14", 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:30.363 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:30.364 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:30.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:30.366 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 6.059999942779541, 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 6.059999942779541, 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.33000001311302185, 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T00:09:30.367 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 2, 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "rbd", 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:49.173060+0000", 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:30.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:30.369 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 8, 2024-09-15T00:09:30.370 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 8, 2024-09-15T00:09:30.370 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 8, 2024-09-15T00:09:30.370 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 8, 2024-09-15T00:09:30.370 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 8, 2024-09-15T00:09:30.370 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:30.370 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:30.370 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:30.370 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:30.370 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "18", 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T00:09:30.371 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 18, 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:30.373 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:30.374 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "pool_client.0", 2024-09-15T00:09:30.375 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:58.879058+0000", 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:30.376 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "46", 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 16, 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 46, 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:30.380 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2024-09-15T00:09:30.381 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:30.382 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "56600dc2-b95d-47a7-b2b1-5ddc54c08341", 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:30.383 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2024-09-15T00:09:30.384 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:30.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.388 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.388 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:30.388 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.389 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:30.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/1440638768", 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/1440638768", 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6823/1440638768", 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6821/1440638768", 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.392 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "ea8f0155-8472-4e87-89fd-7716d05065d0", 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.395 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.396 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.397 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/3601510193", 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/3601510193", 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/3601510193", 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/3601510193", 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "357d2d59-66f0-47b8-87c8-4acdfe62a07d", 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:30.398 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 26, 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-15T00:09:30.399 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.400 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/3222759256", 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/3222759256", 2024-09-15T00:09:30.403 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/3222759256", 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/3222759256", 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "22208f56-d8aa-4a56-a42c-5a990962b860", 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:30.404 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6809", 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:30.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6811", 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6813", 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6816", 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6822", 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6823", 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6818", 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6821", 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6811/675428777", 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6816/675428777", 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6823/675428777", 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6821/675428777", 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:30.409 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "a974629e-9ef0-40a3-94bd-382e0bfa8bb6", 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:30.410 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6808", 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6810", 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6812", 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6814", 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6819", 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6820", 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6815", 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6817", 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6810/3736884138", 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6814/3736884138", 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6820/3736884138", 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6817/3736884138", 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "06b73c2d-466a-4c4e-89fa-31d6eeb1af1d", 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:30.415 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 44, 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6800", 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:30.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6801", 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6802", 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6803", 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6806", 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6807", 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:30.419 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6804", 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:30.420 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6805", 2024-09-15T00:09:30.421 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:30.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.421 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.421 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6801/2235671634", 2024-09-15T00:09:30.421 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6803/2235671634", 2024-09-15T00:09:30.421 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6807/2235671634", 2024-09-15T00:09:30.421 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6805/2235671634", 2024-09-15T00:09:30.421 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:30.422 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.730107+0000", 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:30.423 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.365297+0000", 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.868500+0000", 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.788781+0000", 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:30.425 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.892143+0000", 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.990037+0000", 2024-09-15T00:09:30.426 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.1d", 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 4, 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 2 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.427 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.1e", 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 3, 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 4 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2024-09-15T00:09:30.428 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2024-09-15T00:09:30.429 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2024-09-15T00:09:30.430 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2024-09-15T00:09:30.430 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2024-09-15T00:09:30.430 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:09:30.430 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:09:30.430 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2024-09-15T00:09:30.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.430 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:30.432 INFO:tasks.thrashosds.thrasher:j is {'epoch': 46, 'fsid': '4f532d31-28c1-4236-a74e-6392960baf2b', 'created': '2024-09-15T00:08:32.007191+0000', 'modified': '2024-09-15T00:09:29.888994+0000', 'last_up_change': '2024-09-15T00:08:42.794810+0000', 'last_in_change': '2024-09-15T00:08:33.890963+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:08:42.896076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:08:49.173060+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:08:58.879058+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 46, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '56600dc2-b95d-47a7-b2b1-5ddc54c08341', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 1440638768}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 1440638768}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 1440638768}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 1440638768}]}, 'public_addr': '172.21.15.27:6817/1440638768', 'cluster_addr': '172.21.15.27:6819/1440638768', 'heartbeat_back_addr': '172.21.15.27:6823/1440638768', 'heartbeat_front_addr': '172.21.15.27:6821/1440638768', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ea8f0155-8472-4e87-89fd-7716d05065d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 3601510193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 3601510193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 3601510193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 3601510193}]}, 'public_addr': '172.21.15.27:6809/3601510193', 'cluster_addr': '172.21.15.27:6811/3601510193', 'heartbeat_back_addr': '172.21.15.27:6815/3601510193', 'heartbeat_front_addr': '172.21.15.27:6813/3601510193', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '357d2d59-66f0-47b8-87c8-4acdfe62a07d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 3222759256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 3222759256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 3222759256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 3222759256}]}, 'public_addr': '172.21.15.27:6801/3222759256', 'cluster_addr': '172.21.15.27:6803/3222759256', 'heartbeat_back_addr': '172.21.15.27:6807/3222759256', 'heartbeat_front_addr': '172.21.15.27:6805/3222759256', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '22208f56-d8aa-4a56-a42c-5a990962b860', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6809', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 675428777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6813', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6816', 'nonce': 675428777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 675428777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 675428777}]}, 'public_addr': '172.21.15.79:6811/675428777', 'cluster_addr': '172.21.15.79:6816/675428777', 'heartbeat_back_addr': '172.21.15.79:6823/675428777', 'heartbeat_front_addr': '172.21.15.79:6821/675428777', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a974629e-9ef0-40a3-94bd-382e0bfa8bb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6810', 'nonce': 3736884138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6814', 'nonce': 3736884138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6819', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6820', 'nonce': 3736884138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6815', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 3736884138}]}, 'public_addr': '172.21.15.79:6810/3736884138', 'cluster_addr': '172.21.15.79:6814/3736884138', 'heartbeat_back_addr': '172.21.15.79:6820/3736884138', 'heartbeat_front_addr': '172.21.15.79:6817/3736884138', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '06b73c2d-466a-4c4e-89fa-31d6eeb1af1d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 2235671634}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2235671634}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2235671634}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2235671634}]}, 'public_addr': '172.21.15.79:6801/2235671634', 'cluster_addr': '172.21.15.79:6803/2235671634', 'heartbeat_back_addr': '172.21.15.79:6807/2235671634', 'heartbeat_front_addr': '172.21.15.79:6805/2235671634', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.730107+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.365297+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.868500+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.788781+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.892143+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.990037+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1d', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.1e', 'mappings': [{'from': 3, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:09:30.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:09:30.433 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:30.434 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:30.430+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:30.533 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:30.530+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8832 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3933053149999992, 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056083069999999999, 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8832 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2099999999999998e-07 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153008+0000", 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.187e-06 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153012+0000", 2024-09-15T00:09:30.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8909999999999997e-06 2024-09-15T00:09:30.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153015+0000", 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8559999999999998e-06 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202369+0000", 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.049354393000000003 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202431+0000", 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2235000000000003e-05 2024-09-15T00:09:30.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202749+0000", 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00031714299999999998 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204050+0000", 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00130164 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209033+0000", 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049826590000000004 2024-09-15T00:09:30.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209063+0000", 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0176999999999999e-05 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209089+0000", 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5568e-05 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8833 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3932908360000003, 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056156170999999998, 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8833 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:30.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1299999999999999e-07 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153022+0000", 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5829999999999999e-06 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153025+0000", 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.022e-06 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153029+0000", 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8500000000000004e-06 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203131+0000", 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.05010212 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203216+0000", 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.543e-05 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203493+0000", 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027699000000000002 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206606+0000", 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003112315 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209120+0000", 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002514863 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209149+0000", 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8685000000000001e-05 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209176+0000", 2024-09-15T00:09:30.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7100000000000001e-05 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8834 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3932792939999992, 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056218377999999999, 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:30.562 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:30.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8834 2024-09-15T00:09:30.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.563 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:30.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.16e-07 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153034+0000", 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.844e-06 2024-09-15T00:09:30.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153039+0000", 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8369999999999996e-06 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153041+0000", 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5620000000000002e-06 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204094+0000", 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051053378000000003 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204144+0000", 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9907999999999997e-05 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204342+0000", 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000197596 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206613+0000", 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022710579999999998 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209204+0000", 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0025907780000000002 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209230+0000", 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6041999999999999e-05 2024-09-15T00:09:30.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209250+0000", 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0159000000000001e-05 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.568 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8835 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3932668839999991, 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060077884999999998, 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8835 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.569 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.16e-07 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153046+0000", 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.572 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153049+0000", 2024-09-15T00:09:30.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:09:30.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.572 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.572 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153051+0000", 2024-09-15T00:09:30.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.52e-06 2024-09-15T00:09:30.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204715+0000", 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051664032999999998 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204804+0000", 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8383999999999999e-05 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.573 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:30.574 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205194+0000", 2024-09-15T00:09:30.574 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039034399999999999 2024-09-15T00:09:30.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.574 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:30.574 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206890+0000", 2024-09-15T00:09:30.574 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016954680000000001 2024-09-15T00:09:30.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213068+0000", 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006178411 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213097+0000", 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9433000000000001e-05 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213122+0000", 2024-09-15T00:09:30.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4553999999999999e-05 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8836 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3932564989999996, 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060148357, 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8836 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.577 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153055+0000", 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0100000000000001e-07 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153056+0000", 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6220000000000001e-06 2024-09-15T00:09:30.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153060+0000", 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5499999999999999e-06 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153062+0000", 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6510000000000001e-06 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205724+0000", 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.052661396999999999 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205802+0000", 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8668e-05 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206109+0000", 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030663400000000002 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.208482+0000", 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023725019999999999 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213151+0000", 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046689660000000001 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213181+0000", 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0324999999999998e-05 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213203+0000", 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1841e-05 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8837 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3932456260000006, 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060386983999999998, 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8837 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1500000000000001e-07 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153067+0000", 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.764e-06 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153070+0000", 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.869e-06 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153073+0000", 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.548e-06 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206907+0000", 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053834290999999999 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206953+0000", 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5648000000000001e-05 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207282+0000", 2024-09-15T00:09:30.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032939999999999998 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209097+0000", 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018149170000000001 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213388+0000", 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042910989999999996 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213430+0000", 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2465000000000001e-05 2024-09-15T00:09:30.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213452+0000", 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1767999999999999e-05 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8838 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3932353840000005, 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060498379999999997, 2024-09-15T00:09:30.588 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8838 2024-09-15T00:09:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:30.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.04e-07 2024-09-15T00:09:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153077+0000", 2024-09-15T00:09:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.389e-06 2024-09-15T00:09:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153080+0000", 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8499999999999998e-06 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153083+0000", 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.565e-06 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207872+0000", 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.054789110000000002 2024-09-15T00:09:30.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207926+0000", 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4515000000000002e-05 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.208143+0000", 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000216784 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209511+0000", 2024-09-15T00:09:30.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013684350000000001 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213494+0000", 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039824889999999996 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213535+0000", 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0865999999999998e-05 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213574+0000", 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9172999999999999e-05 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3358 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7684096~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.50014100100000003, 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055739138000000001, 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3358 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046177+0000", 2024-09-15T00:09:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7000000000000002e-06 2024-09-15T00:09:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046184+0000", 2024-09-15T00:09:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.029e-06 2024-09-15T00:09:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046189+0000", 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9459999999999997e-06 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101686+0000", 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055497367999999998 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101778+0000", 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2077000000000006e-05 2024-09-15T00:09:30.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101909+0000", 2024-09-15T00:09:30.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013064700000000001 2024-09-15T00:09:30.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.602 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.603 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3359 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7733248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.603 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:30.603 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.50011325699999998, 2024-09-15T00:09:30.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055953638999999999, 2024-09-15T00:09:30.603 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.603 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3359 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046200+0000", 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046207+0000", 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8050000000000001e-06 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046213+0000", 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3489999999999999e-06 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101957+0000", 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055744316000000002 2024-09-15T00:09:30.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102048+0000", 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1464000000000004e-05 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102151+0000", 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102874 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3360 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7782400~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.50009084599999998, 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056128863000000001, 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3360 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.608 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:30.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046223+0000", 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046229+0000", 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6839999999999999e-06 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046234+0000", 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6800000000000001e-06 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102187+0000", 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.05595282 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102262+0000", 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5371999999999996e-05 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102349+0000", 2024-09-15T00:09:30.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6706999999999995e-05 2024-09-15T00:09:30.612 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.612 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.612 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.612 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3361 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7831552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.612 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.50007001200000001, 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056322561, 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3361 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046244+0000", 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.43e-06 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046250+0000", 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5420000000000002e-06 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046255+0000", 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.707e-06 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102382+0000", 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056126796999999999 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102469+0000", 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7023999999999999e-05 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102563+0000", 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4678000000000004e-05 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3362 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7880704~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046262+0000", 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.50004897000000004, 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056530960999999998, 2024-09-15T00:09:30.617 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3362 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046262+0000", 2024-09-15T00:09:30.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046262+0000", 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046492+0000", 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023003100000000001 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046496+0000", 2024-09-15T00:09:30.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9210000000000002e-06 2024-09-15T00:09:30.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046506+0000", 2024-09-15T00:09:30.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0001e-05 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046513+0000", 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9229999999999996e-06 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102610+0000", 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056097039000000001 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102698+0000", 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7860000000000002e-05 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102793+0000", 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5186000000000006e-05 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3363 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7929856~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046548+0000", 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.49976273500000001, 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056535131000000002, 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3363 2024-09-15T00:09:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046548+0000", 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046548+0000", 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046549+0000", 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6000000000000003e-07 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046552+0000", 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046564+0000", 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1474999999999999e-05 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046572+0000", 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4009999999999997e-06 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102824+0000", 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056251511999999997 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102922+0000", 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8499999999999995e-05 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103083+0000", 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016119300000000001 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3364 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7979008~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046606+0000", 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.499704488, 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056797145, 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3364 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046606+0000", 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046606+0000", 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047399+0000", 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00079283299999999995 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047403+0000", 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1670000000000001e-06 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047413+0000", 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4220000000000006e-06 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047419+0000", 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4450000000000001e-06 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103137+0000", 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055717322999999999 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103252+0000", 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115701 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103404+0000", 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000151254 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3365 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8028160~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.047431+0000", 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.498879882, 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056539155000000001, 2024-09-15T00:09:30.632 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3365 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047431+0000", 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047431+0000", 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047432+0000", 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047434+0000", 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:09:30.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047441+0000", 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.849e-06 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047447+0000", 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.609e-06 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103694+0000", 2024-09-15T00:09:30.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056247765999999998 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103808+0000", 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011345699999999999 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103970+0000", 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000162421 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3366 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8077312~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.047453+0000", 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.49885743700000001, 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056741575000000002, 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3366 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047453+0000", 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047453+0000", 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047454+0000", 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047456+0000", 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.125e-06 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047462+0000", 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5999999999999997e-06 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047466+0000", 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5990000000000004e-06 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104001+0000", 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056535215 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104093+0000", 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1496999999999995e-05 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104195+0000", 2024-09-15T00:09:30.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010213900000000001 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3367 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8126464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.047473+0000", 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.49883816199999997, 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056984806999999998, 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.642 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3367 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047473+0000", 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047473+0000", 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049398+0000", 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001925689 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049403+0000", 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7700000000000001e-06 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049415+0000", 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1657999999999999e-05 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049422+0000", 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6350000000000006e-06 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104236+0000", 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.054813957000000003 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104352+0000", 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011547100000000001 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104458+0000", 2024-09-15T00:09:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105627 2024-09-15T00:09:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3370 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8273920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:30.648 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.496792128, 2024-09-15T00:09:30.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055967186000000002, 2024-09-15T00:09:30.648 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.648 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.648 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3370 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9999999999999997e-07 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049522+0000", 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5900000000000002e-06 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049531+0000", 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0270000000000001e-06 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049537+0000", 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8610000000000003e-06 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105279+0000", 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055742236000000001 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105364+0000", 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4993000000000004e-05 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105486+0000", 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.652 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3371 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8323072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.049545+0000", 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.49676559100000001, 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056146330000000001, 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3371 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.653 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049545+0000", 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049545+0000", 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049546+0000", 2024-09-15T00:09:30.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7300000000000001e-07 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049548+0000", 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5770000000000001e-06 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049740+0000", 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019119300000000001 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049749+0000", 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3850000000000008e-06 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105519+0000", 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055770239999999999 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105601+0000", 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1460999999999998e-05 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105692+0000", 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1001000000000005e-05 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3372 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8372224~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.49654622599999998, 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056196995, 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:30.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:30.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:30.658 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:30.658 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:30.658 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:30.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3372 2024-09-15T00:09:30.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.658 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:30.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:30.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:30.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:30.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:30.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:30.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:30.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:30.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9100000000000004e-07 2024-09-15T00:09:30.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:30.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049769+0000", 2024-09-15T00:09:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8399999999999997e-06 2024-09-15T00:09:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049777+0000", 2024-09-15T00:09:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:09:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049783+0000", 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105753+0000", 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055970585000000003 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105831+0000", 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7171999999999999e-05 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:30.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:30.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:30.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105962+0000", 2024-09-15T00:09:30.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013104800000000001 2024-09-15T00:09:30.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.663 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:30.663 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:30.663 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:30.665 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:30.665 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:30.665 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:30.666 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:30.666 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:30.666 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:30.667 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:30.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:30.667 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:30.667 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:30.663+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:30.673 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:30.673 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:30.689 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:30.765 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:30.763+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:30.774 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:09:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:09:30.782 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:30.782 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:30.788 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-15T00:09:29.893401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":499232371,"num_objects":525,"num_object_clones":124,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":105,"num_read":73921,"num_read_kb":358430,"num_write":6002,"num_write_kb":70382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4722,"ondisk_log_size":4722,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":221416,"kb_used_data":17640,"kb_used_omap":9,"kb_used_meta":203638,"kb_avail":566009624,"statfs":{"total":579820584960,"available":579593854976,"internally_reserved":0,"allocated":18063360,"data_stored":101844915,"data_compressed":1968964,"data_compressed_allocated":6832128,"data_compressed_original":93623296,"omap_allocated":9568,"internal_metadata":208525984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":105857612,"num_objects":30,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":4103,"num_read_kb":23497,"num_write":142,"num_write_kb":3720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018463"},"pg_stats":[{"pgid":"3.1f","version":"39'160","reported_seq":2920,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.888546+0000","last_change":"2024-09-15T00:09:01.970297+0000","last_active":"2024-09-15T00:09:26.888546+0000","last_peered":"2024-09-15T00:09:26.888546+0000","last_clean":"2024-09-15T00:09:26.888546+0000","last_became_active":"2024-09-15T00:08:59.945793+0000","last_became_peered":"2024-09-15T00:08:59.945793+0000","last_unstale":"2024-09-15T00:09:26.888546+0000","last_undegraded":"2024-09-15T00:09:26.888546+0000","last_fullsized":"2024-09-15T00:09:26.888546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:53.892803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075174400000000004,"stat_sum":{"num_bytes":18440720,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":3687,"num_read_kb":14105,"num_write":147,"num_write_kb":879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"40'68","reported_seq":1645,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.086943+0000","last_change":"2024-09-15T00:09:06.797902+0000","last_active":"2024-09-15T00:09:27.086943+0000","last_peered":"2024-09-15T00:09:27.086943+0000","last_clean":"2024-09-15T00:09:27.086943+0000","last_became_active":"2024-09-15T00:09:06.795735+0000","last_became_peered":"2024-09-15T00:09:06.795735+0000","last_unstale":"2024-09-15T00:09:27.086943+0000","last_undegraded":"2024-09-15T00:09:27.086943+0000","last_fullsized":"2024-09-15T00:09:27.086943+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:19:43.985220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031880599999999998,"stat_sum":{"num_bytes":15587882,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1531,"num_read_kb":13981,"num_write":70,"num_write_kb":2731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"38'166","reported_seq":1480,"reported_epoch":43,"state":"active+undersized+remapped","last_fresh":"2024-09-15T00:09:26.937437+0000","last_change":"2024-09-15T00:09:26.937682+0000","last_active":"2024-09-15T00:09:26.937437+0000","last_peered":"2024-09-15T00:09:26.937437+0000","last_clean":"2024-09-15T00:09:24.555863+0000","last_became_active":"2024-09-15T00:09:26.884509+0000","last_became_peered":"2024-09-15T00:09:26.884509+0000","last_unstale":"2024-09-15T00:09:26.937437+0000","last_undegraded":"2024-09-15T00:09:26.937437+0000","last_fullsized":"2024-09-15T00:09:26.879345+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001450004,"stat_sum":{"num_bytes":14455847,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2282,"num_read_kb":2865,"num_write":169,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5],"avail_no_missing":["5","2","4"],"object_location_counts":[{"shards":"5","objects":17}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"43'195","reported_seq":3187,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.267750+0000","last_change":"2024-09-15T00:09:01.964719+0000","last_active":"2024-09-15T00:09:27.267750+0000","last_peered":"2024-09-15T00:09:27.267750+0000","last_clean":"2024-09-15T00:09:27.267750+0000","last_became_active":"2024-09-15T00:08:59.945681+0000","last_became_peered":"2024-09-15T00:08:59.945681+0000","last_unstale":"2024-09-15T00:09:27.267750+0000","last_undegraded":"2024-09-15T00:09:27.267750+0000","last_fullsized":"2024-09-15T00:09:27.267750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:59:10.454301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090320299999999999,"stat_sum":{"num_bytes":21197861,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":3474,"num_read_kb":24821,"num_write":289,"num_write_kb":3973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"43'121","reported_seq":3353,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.243315+0000","last_change":"2024-09-15T00:09:01.969947+0000","last_active":"2024-09-15T00:09:27.243315+0000","last_peered":"2024-09-15T00:09:27.243315+0000","last_clean":"2024-09-15T00:09:27.243315+0000","last_became_active":"2024-09-15T00:08:59.945617+0000","last_became_peered":"2024-09-15T00:08:59.945617+0000","last_unstale":"2024-09-15T00:09:27.243315+0000","last_undegraded":"2024-09-15T00:09:27.243315+0000","last_fullsized":"2024-09-15T00:09:27.243315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:15:41.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048451499999999998,"stat_sum":{"num_bytes":41065050,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":3375,"num_read_kb":22942,"num_write":171,"num_write_kb":1656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"39'133","reported_seq":4722,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.275685+0000","last_change":"2024-09-15T00:09:01.961084+0000","last_active":"2024-09-15T00:09:27.275685+0000","last_peered":"2024-09-15T00:09:27.275685+0000","last_clean":"2024-09-15T00:09:27.275685+0000","last_became_active":"2024-09-15T00:08:59.946828+0000","last_became_peered":"2024-09-15T00:08:59.946828+0000","last_unstale":"2024-09-15T00:09:27.275685+0000","last_undegraded":"2024-09-15T00:09:27.275685+0000","last_fullsized":"2024-09-15T00:09:27.275685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:19:05.674956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085056300000000004,"stat_sum":{"num_bytes":12448327,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":2,"num_read":5537,"num_read_kb":19018,"num_write":119,"num_write_kb":4125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"43'120","reported_seq":1315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.227038+0000","last_change":"2024-09-15T00:09:01.958081+0000","last_active":"2024-09-15T00:09:27.227038+0000","last_peered":"2024-09-15T00:09:27.227038+0000","last_clean":"2024-09-15T00:09:27.227038+0000","last_became_active":"2024-09-15T00:08:59.942052+0000","last_became_peered":"2024-09-15T00:08:59.942052+0000","last_unstale":"2024-09-15T00:09:27.227038+0000","last_undegraded":"2024-09-15T00:09:27.227038+0000","last_fullsized":"2024-09-15T00:09:27.227038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:45:25.193853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055705900000000005,"stat_sum":{"num_bytes":7670272,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1617,"num_read_kb":5072,"num_write":130,"num_write_kb":1213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"43'307","reported_seq":3869,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.300046+0000","last_change":"2024-09-15T00:09:01.978690+0000","last_active":"2024-09-15T00:09:27.300046+0000","last_peered":"2024-09-15T00:09:27.300046+0000","last_clean":"2024-09-15T00:09:27.300046+0000","last_became_active":"2024-09-15T00:08:59.941686+0000","last_became_peered":"2024-09-15T00:08:59.941686+0000","last_unstale":"2024-09-15T00:09:27.300046+0000","last_undegraded":"2024-09-15T00:09:27.300046+0000","last_fullsized":"2024-09-15T00:09:27.300046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:52:15.443653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000308933,"stat_sum":{"num_bytes":63858206,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":5383,"num_read_kb":25046,"num_write":362,"num_write_kb":5603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"38'118","reported_seq":3126,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.086035+0000","last_change":"2024-09-15T00:09:01.970454+0000","last_active":"2024-09-15T00:09:27.086035+0000","last_peered":"2024-09-15T00:09:27.086035+0000","last_clean":"2024-09-15T00:09:27.086035+0000","last_became_active":"2024-09-15T00:08:59.946139+0000","last_became_peered":"2024-09-15T00:08:59.946139+0000","last_unstale":"2024-09-15T00:09:27.086035+0000","last_undegraded":"2024-09-15T00:09:27.086035+0000","last_fullsized":"2024-09-15T00:09:27.086035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:11:07.132331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084960000000000005,"stat_sum":{"num_bytes":8395776,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":3456,"num_read_kb":13748,"num_write":98,"num_write_kb":1365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"43'82","reported_seq":2433,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.210059+0000","last_change":"2024-09-15T00:09:01.965029+0000","last_active":"2024-09-15T00:09:27.210059+0000","last_peered":"2024-09-15T00:09:27.210059+0000","last_clean":"2024-09-15T00:09:27.210059+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:09:27.210059+0000","last_undegraded":"2024-09-15T00:09:27.210059+0000","last_fullsized":"2024-09-15T00:09:27.210059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:15:39.607365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062487,"stat_sum":{"num_bytes":28260864,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":2309,"num_read_kb":10041,"num_write":107,"num_write_kb":2987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"39'146","reported_seq":1950,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.086074+0000","last_change":"2024-09-15T00:09:01.953424+0000","last_active":"2024-09-15T00:09:27.086074+0000","last_peered":"2024-09-15T00:09:27.086074+0000","last_clean":"2024-09-15T00:09:27.086074+0000","last_became_active":"2024-09-15T00:08:59.941565+0000","last_became_peered":"2024-09-15T00:08:59.941565+0000","last_unstale":"2024-09-15T00:09:27.086074+0000","last_undegraded":"2024-09-15T00:09:27.086074+0000","last_fullsized":"2024-09-15T00:09:27.086074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:33.469940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044338899999999998,"stat_sum":{"num_bytes":6141952,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1972,"num_read_kb":4874,"num_write":141,"num_write_kb":812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"40'55","reported_seq":592,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.878033+0000","last_change":"2024-09-15T00:09:01.953831+0000","last_active":"2024-09-15T00:09:26.878033+0000","last_peered":"2024-09-15T00:09:26.878033+0000","last_clean":"2024-09-15T00:09:26.878033+0000","last_became_active":"2024-09-15T00:08:59.942776+0000","last_became_peered":"2024-09-15T00:08:59.942776+0000","last_unstale":"2024-09-15T00:09:26.878033+0000","last_undegraded":"2024-09-15T00:09:26.878033+0000","last_fullsized":"2024-09-15T00:09:26.878033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:47:00.619224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079073999999999995,"stat_sum":{"num_bytes":6951952,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":484,"num_read_kb":4453,"num_write":60,"num_write_kb":532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.878031+0000","last_change":"2024-09-15T00:08:52.835701+0000","last_active":"2024-09-15T00:09:26.878031+0000","last_peered":"2024-09-15T00:09:26.878031+0000","last_clean":"2024-09-15T00:09:26.878031+0000","last_became_active":"2024-09-15T00:08:50.969842+0000","last_became_peered":"2024-09-15T00:08:50.969842+0000","last_unstale":"2024-09-15T00:09:26.878031+0000","last_undegraded":"2024-09-15T00:09:26.878031+0000","last_fullsized":"2024-09-15T00:09:26.878031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:00.926947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"39'69","reported_seq":740,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.053131+0000","last_change":"2024-09-15T00:09:01.970380+0000","last_active":"2024-09-15T00:09:27.053131+0000","last_peered":"2024-09-15T00:09:27.053131+0000","last_clean":"2024-09-15T00:09:27.053131+0000","last_became_active":"2024-09-15T00:08:59.946388+0000","last_became_peered":"2024-09-15T00:08:59.946388+0000","last_unstale":"2024-09-15T00:09:27.053131+0000","last_undegraded":"2024-09-15T00:09:27.053131+0000","last_fullsized":"2024-09-15T00:09:27.053131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:29.508593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083591999999999998,"stat_sum":{"num_bytes":6364672,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":617,"num_read_kb":3414,"num_write":66,"num_write_kb":616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.888365+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:09:26.888365+0000","last_peered":"2024-09-15T00:09:26.888365+0000","last_clean":"2024-09-15T00:09:26.888365+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:09:26.888365+0000","last_undegraded":"2024-09-15T00:09:26.888365+0000","last_fullsized":"2024-09-15T00:09:26.888365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"38'52","reported_seq":667,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.086169+0000","last_change":"2024-09-15T00:09:01.979030+0000","last_active":"2024-09-15T00:09:27.086169+0000","last_peered":"2024-09-15T00:09:27.086169+0000","last_clean":"2024-09-15T00:09:27.086169+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:09:27.086169+0000","last_undegraded":"2024-09-15T00:09:27.086169+0000","last_fullsized":"2024-09-15T00:09:27.086169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:14:43.811661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059204099999999996,"stat_sum":{"num_bytes":7578158,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":571,"num_read_kb":4464,"num_write":52,"num_write_kb":569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.878208+0000","last_change":"2024-09-15T00:08:52.835997+0000","last_active":"2024-09-15T00:09:26.878208+0000","last_peered":"2024-09-15T00:09:26.878208+0000","last_clean":"2024-09-15T00:09:26.878208+0000","last_became_active":"2024-09-15T00:08:51.070763+0000","last_became_peered":"2024-09-15T00:08:51.070763+0000","last_unstale":"2024-09-15T00:09:26.878208+0000","last_undegraded":"2024-09-15T00:09:26.878208+0000","last_fullsized":"2024-09-15T00:09:26.878208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:53:58.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'345","reported_seq":3154,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.297243+0000","last_change":"2024-09-15T00:09:01.958574+0000","last_active":"2024-09-15T00:09:27.297243+0000","last_peered":"2024-09-15T00:09:27.297243+0000","last_clean":"2024-09-15T00:09:27.297243+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:09:27.297243+0000","last_undegraded":"2024-09-15T00:09:27.297243+0000","last_fullsized":"2024-09-15T00:09:27.297243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:19.324269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070311699999999996,"stat_sum":{"num_bytes":32839245,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":5521,"num_read_kb":10865,"num_write":365,"num_write_kb":3563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.889001+0000","last_change":"2024-09-15T00:08:52.842966+0000","last_active":"2024-09-15T00:09:26.889001+0000","last_peered":"2024-09-15T00:09:26.889001+0000","last_clean":"2024-09-15T00:09:26.889001+0000","last_became_active":"2024-09-15T00:08:51.070528+0000","last_became_peered":"2024-09-15T00:08:51.070528+0000","last_unstale":"2024-09-15T00:09:26.889001+0000","last_undegraded":"2024-09-15T00:09:26.889001+0000","last_fullsized":"2024-09-15T00:09:26.889001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:52:05.164188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"43'107","reported_seq":1718,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.269630+0000","last_change":"2024-09-15T00:09:01.965506+0000","last_active":"2024-09-15T00:09:27.269630+0000","last_peered":"2024-09-15T00:09:27.269630+0000","last_clean":"2024-09-15T00:09:27.269630+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:09:27.269630+0000","last_undegraded":"2024-09-15T00:09:27.269630+0000","last_fullsized":"2024-09-15T00:09:27.269630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:04:24.295603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001375294,"stat_sum":{"num_bytes":5381700,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":1638,"num_read_kb":8526,"num_write":112,"num_write_kb":925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"39'79","reported_seq":1435,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.882448+0000","last_change":"2024-09-15T00:09:01.960514+0000","last_active":"2024-09-15T00:09:26.882448+0000","last_peered":"2024-09-15T00:09:26.882448+0000","last_clean":"2024-09-15T00:09:26.882448+0000","last_became_active":"2024-09-15T00:08:59.944404+0000","last_became_peered":"2024-09-15T00:08:59.944404+0000","last_unstale":"2024-09-15T00:09:26.882448+0000","last_undegraded":"2024-09-15T00:09:26.882448+0000","last_fullsized":"2024-09-15T00:09:26.882448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:43:44.398509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037381900000000001,"stat_sum":{"num_bytes":8480807,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":1314,"num_read_kb":9141,"num_write":116,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.879204+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:09:26.879204+0000","last_peered":"2024-09-15T00:09:26.879204+0000","last_clean":"2024-09-15T00:09:26.879204+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:09:26.879204+0000","last_undegraded":"2024-09-15T00:09:26.879204+0000","last_fullsized":"2024-09-15T00:09:26.879204+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"39'277","reported_seq":766,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.053417+0000","last_change":"2024-09-15T00:09:01.958075+0000","last_active":"2024-09-15T00:09:27.053417+0000","last_peered":"2024-09-15T00:09:27.053417+0000","last_clean":"2024-09-15T00:09:27.053417+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:09:27.053417+0000","last_undegraded":"2024-09-15T00:09:27.053417+0000","last_fullsized":"2024-09-15T00:09:27.053417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:56.935095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000567013,"stat_sum":{"num_bytes":5413509,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1630,"num_read_kb":3886,"num_write":463,"num_write_kb":1796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.877508+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:09:26.877508+0000","last_peered":"2024-09-15T00:09:26.877508+0000","last_clean":"2024-09-15T00:09:26.877508+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:09:26.877508+0000","last_undegraded":"2024-09-15T00:09:26.877508+0000","last_fullsized":"2024-09-15T00:09:26.877508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"39'84","reported_seq":745,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.888478+0000","last_change":"2024-09-15T00:09:01.969727+0000","last_active":"2024-09-15T00:09:26.888478+0000","last_peered":"2024-09-15T00:09:26.888478+0000","last_clean":"2024-09-15T00:09:26.888478+0000","last_became_active":"2024-09-15T00:08:59.942545+0000","last_became_peered":"2024-09-15T00:08:59.942545+0000","last_unstale":"2024-09-15T00:09:26.888478+0000","last_undegraded":"2024-09-15T00:09:26.888478+0000","last_fullsized":"2024-09-15T00:09:26.888478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:29:28.085226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000355144,"stat_sum":{"num_bytes":6495280,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":816,"num_read_kb":5969,"num_write":116,"num_write_kb":1052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.877780+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:09:26.877780+0000","last_peered":"2024-09-15T00:09:26.877780+0000","last_clean":"2024-09-15T00:09:26.877780+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:09:26.877780+0000","last_undegraded":"2024-09-15T00:09:26.877780+0000","last_fullsized":"2024-09-15T00:09:26.877780+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"43'174","reported_seq":3876,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.270883+0000","last_change":"2024-09-15T00:09:01.958272+0000","last_active":"2024-09-15T00:09:27.270883+0000","last_peered":"2024-09-15T00:09:27.270883+0000","last_clean":"2024-09-15T00:09:27.270883+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:09:27.270883+0000","last_undegraded":"2024-09-15T00:09:27.270883+0000","last_fullsized":"2024-09-15T00:09:27.270883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:38:41.374383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056893999999999998,"stat_sum":{"num_bytes":10457616,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":3539,"num_read_kb":10912,"num_write":124,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.879244+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:09:26.879244+0000","last_peered":"2024-09-15T00:09:26.879244+0000","last_clean":"2024-09-15T00:09:26.879244+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:09:26.879244+0000","last_undegraded":"2024-09-15T00:09:26.879244+0000","last_fullsized":"2024-09-15T00:09:26.879244+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":219,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.877820+0000","last_change":"2024-09-15T00:08:44.804415+0000","last_active":"2024-09-15T00:09:26.877820+0000","last_peered":"2024-09-15T00:09:26.877820+0000","last_clean":"2024-09-15T00:09:26.877820+0000","last_became_active":"2024-09-15T00:08:44.804011+0000","last_became_peered":"2024-09-15T00:08:44.804011+0000","last_unstale":"2024-09-15T00:09:26.877820+0000","last_undegraded":"2024-09-15T00:09:26.877820+0000","last_fullsized":"2024-09-15T00:09:26.877820+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:12.388733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"40'108","reported_seq":2491,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.086773+0000","last_change":"2024-09-15T00:09:01.978947+0000","last_active":"2024-09-15T00:09:27.086773+0000","last_peered":"2024-09-15T00:09:27.086773+0000","last_clean":"2024-09-15T00:09:27.086773+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:09:27.086773+0000","last_undegraded":"2024-09-15T00:09:27.086773+0000","last_fullsized":"2024-09-15T00:09:27.086773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:57:57.285771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051427500000000002,"stat_sum":{"num_bytes":34891792,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":2653,"num_read_kb":25522,"num_write":155,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"39'106","reported_seq":4114,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.085968+0000","last_change":"2024-09-15T00:09:01.960508+0000","last_active":"2024-09-15T00:09:27.085968+0000","last_peered":"2024-09-15T00:09:27.085968+0000","last_clean":"2024-09-15T00:09:27.085968+0000","last_became_active":"2024-09-15T00:08:59.945869+0000","last_became_peered":"2024-09-15T00:08:59.945869+0000","last_unstale":"2024-09-15T00:09:27.085968+0000","last_undegraded":"2024-09-15T00:09:27.085968+0000","last_fullsized":"2024-09-15T00:09:27.085968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:38:05.527204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031568600000000003,"stat_sum":{"num_bytes":28652544,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":3978,"num_read_kb":27425,"num_write":130,"num_write_kb":5306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"40'72","reported_seq":751,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.086596+0000","last_change":"2024-09-15T00:09:01.953327+0000","last_active":"2024-09-15T00:09:27.086596+0000","last_peered":"2024-09-15T00:09:27.086596+0000","last_clean":"2024-09-15T00:09:27.086596+0000","last_became_active":"2024-09-15T00:08:59.942796+0000","last_became_peered":"2024-09-15T00:08:59.942796+0000","last_unstale":"2024-09-15T00:09:27.086596+0000","last_undegraded":"2024-09-15T00:09:27.086596+0000","last_fullsized":"2024-09-15T00:09:27.086596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:07.907823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035268300000000001,"stat_sum":{"num_bytes":2836627,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":656,"num_read_kb":4029,"num_write":71,"num_write_kb":551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"40'69","reported_seq":942,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.888745+0000","last_change":"2024-09-15T00:09:01.958373+0000","last_active":"2024-09-15T00:09:26.888745+0000","last_peered":"2024-09-15T00:09:26.888745+0000","last_clean":"2024-09-15T00:09:26.888745+0000","last_became_active":"2024-09-15T00:08:59.941757+0000","last_became_peered":"2024-09-15T00:08:59.941757+0000","last_unstale":"2024-09-15T00:09:26.888745+0000","last_undegraded":"2024-09-15T00:09:26.888745+0000","last_fullsized":"2024-09-15T00:09:26.888745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:14:28.501332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071565300000000001,"stat_sum":{"num_bytes":12867674,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":832,"num_read_kb":7413,"num_write":67,"num_write_kb":414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"40'154","reported_seq":1001,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.258008+0000","last_change":"2024-09-15T00:09:01.964756+0000","last_active":"2024-09-15T00:09:27.258008+0000","last_peered":"2024-09-15T00:09:27.258008+0000","last_clean":"2024-09-15T00:09:27.258008+0000","last_became_active":"2024-09-15T00:08:59.946686+0000","last_became_peered":"2024-09-15T00:08:59.946686+0000","last_unstale":"2024-09-15T00:09:27.258008+0000","last_undegraded":"2024-09-15T00:09:27.258008+0000","last_fullsized":"2024-09-15T00:09:27.258008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:23:15.800369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092954400000000003,"stat_sum":{"num_bytes":9096228,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1195,"num_read_kb":4287,"num_write":201,"num_write_kb":1991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"40'63","reported_seq":894,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.239204+0000","last_change":"2024-09-15T00:09:01.953837+0000","last_active":"2024-09-15T00:09:27.239204+0000","last_peered":"2024-09-15T00:09:27.239204+0000","last_clean":"2024-09-15T00:09:27.239204+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:09:27.239204+0000","last_undegraded":"2024-09-15T00:09:27.239204+0000","last_fullsized":"2024-09-15T00:09:27.239204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:56:46.986520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078421999999999997,"stat_sum":{"num_bytes":11124775,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":795,"num_read_kb":6402,"num_write":71,"num_write_kb":1313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"39'110","reported_seq":2397,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.053396+0000","last_change":"2024-09-15T00:09:01.965099+0000","last_active":"2024-09-15T00:09:27.053396+0000","last_peered":"2024-09-15T00:09:27.053396+0000","last_clean":"2024-09-15T00:09:27.053396+0000","last_became_active":"2024-09-15T00:08:59.945750+0000","last_became_peered":"2024-09-15T00:08:59.945750+0000","last_unstale":"2024-09-15T00:09:27.053396+0000","last_undegraded":"2024-09-15T00:09:27.053396+0000","last_fullsized":"2024-09-15T00:09:27.053396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:58:23.929407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010923459999999999,"stat_sum":{"num_bytes":23212080,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":2273,"num_read_kb":14024,"num_write":112,"num_write_kb":809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"43'78","reported_seq":556,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.291501+0000","last_change":"2024-09-15T00:09:01.965290+0000","last_active":"2024-09-15T00:09:27.291501+0000","last_peered":"2024-09-15T00:09:27.291501+0000","last_clean":"2024-09-15T00:09:27.291501+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:09:27.291501+0000","last_undegraded":"2024-09-15T00:09:27.291501+0000","last_fullsized":"2024-09-15T00:09:27.291501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:29:01.900281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001242525,"stat_sum":{"num_bytes":3371024,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":537,"num_read_kb":1245,"num_write":72,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"40'80","reported_seq":1530,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.014753+0000","last_change":"2024-09-15T00:09:01.964202+0000","last_active":"2024-09-15T00:09:27.014753+0000","last_peered":"2024-09-15T00:09:27.014753+0000","last_clean":"2024-09-15T00:09:27.014753+0000","last_became_active":"2024-09-15T00:08:59.945540+0000","last_became_peered":"2024-09-15T00:08:59.945540+0000","last_unstale":"2024-09-15T00:09:27.014753+0000","last_undegraded":"2024-09-15T00:09:27.014753+0000","last_fullsized":"2024-09-15T00:09:27.014753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:41.813852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041955600000000002,"stat_sum":{"num_bytes":8314368,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":1406,"num_read_kb":11583,"num_write":114,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"40'65","reported_seq":1159,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:26.964709+0000","last_change":"2024-09-15T00:09:01.953573+0000","last_active":"2024-09-15T00:09:26.964709+0000","last_peered":"2024-09-15T00:09:26.964709+0000","last_clean":"2024-09-15T00:09:26.964709+0000","last_became_active":"2024-09-15T00:08:59.942379+0000","last_became_peered":"2024-09-15T00:08:59.942379+0000","last_unstale":"2024-09-15T00:09:26.964709+0000","last_undegraded":"2024-09-15T00:09:26.964709+0000","last_fullsized":"2024-09-15T00:09:26.964709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:15.655552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067808099999999997,"stat_sum":{"num_bytes":5399552,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":1037,"num_read_kb":6074,"num_write":78,"num_write_kb":799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"39'767","reported_seq":3498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.053089+0000","last_change":"2024-09-15T00:09:01.969823+0000","last_active":"2024-09-15T00:09:27.053089+0000","last_peered":"2024-09-15T00:09:27.053089+0000","last_clean":"2024-09-15T00:09:27.053089+0000","last_became_active":"2024-09-15T00:08:59.946308+0000","last_became_peered":"2024-09-15T00:08:59.946308+0000","last_unstale":"2024-09-15T00:09:27.053089+0000","last_undegraded":"2024-09-15T00:09:27.053089+0000","last_fullsized":"2024-09-15T00:09:27.053089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:15.582903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041010299999999999,"stat_sum":{"num_bytes":25597826,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":3,"num_read":6656,"num_read_kb":28139,"num_write":1426,"num_write_kb":5103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"39'72","reported_seq":1146,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T00:09:27.014787+0000","last_change":"2024-09-15T00:09:01.953644+0000","last_active":"2024-09-15T00:09:27.014787+0000","last_peered":"2024-09-15T00:09:27.014787+0000","last_clean":"2024-09-15T00:09:27.014787+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:09:27.014787+0000","last_undegraded":"2024-09-15T00:09:27.014787+0000","last_fullsized":"2024-09-15T00:09:27.014787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:29:41.345246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061215200000000003,"stat_sum":{"num_bytes":5791798,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":1034,"num_read_kb":4044,"num_write":67,"num_write_kb":563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":498641984,"num_objects":521,"num_object_clones":124,"num_object_copies":1042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":521,"num_whiteouts":105,"num_read":73805,"num_read_kb":358330,"num_write":5841,"num_write_kb":67964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15278080,"data_stored":99736231,"data_compressed":1951908,"data_compressed_allocated":6758400,"data_compressed_original":92476416,"omap_allocated":0,"internal_metadata":0},"log_size":4602,"ondisk_log_size":4602,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17056,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640268,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":2490368,"data_stored":9485482,"data_compressed":348449,"data_compressed_allocated":782336,"data_compressed_original":8256512,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640268,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34548,"kb_used_data":3356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337292,"statfs":{"total":96636764160,"available":96601387008,"internally_reserved":0,"allocated":3436544,"data_stored":19520534,"data_compressed":295456,"data_compressed_allocated":1216512,"data_compressed_original":17821184,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640269,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37512,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334328,"statfs":{"total":96636764160,"available":96598351872,"internally_reserved":0,"allocated":2211840,"data_stored":8197663,"data_compressed":222377,"data_compressed_allocated":618496,"data_compressed_original":7074816,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640268,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39256,"kb_used_data":3912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332584,"statfs":{"total":96636764160,"available":96596566016,"internally_reserved":0,"allocated":4005888,"data_stored":31105863,"data_compressed":544209,"data_compressed_allocated":1921024,"data_compressed_original":29556736,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640268,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2641920,"data_stored":17087793,"data_compressed":288707,"data_compressed_allocated":1089536,"data_compressed_original":15996416,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640269,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38548,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333292,"statfs":{"total":96636764160,"available":96597291008,"internally_reserved":0,"allocated":3276800,"data_stored":16447580,"data_compressed":269766,"data_compressed_allocated":1204224,"data_compressed_original":14917632,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2740224,"data_stored":15671519,"data_compressed":261238,"data_compressed_allocated":1167360,"data_compressed_original":14344192,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2158592,"data_stored":16902081,"data_compressed":288707,"data_compressed_allocated":1089536,"data_compressed_original":15996416,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3526656,"data_stored":30920170,"data_compressed":544209,"data_compressed_allocated":1921024,"data_compressed_original":29556736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1675264,"data_stored":7421602,"data_compressed":213849,"data_compressed_allocated":581632,"data_compressed_original":6501376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3170304,"data_stored":19521089,"data_compressed":295456,"data_compressed_allocated":1216512,"data_compressed_original":17821184,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2007040,"data_stored":9299770,"data_compressed":348449,"data_compressed_allocated":782336,"data_compressed_original":8256512,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:09:30.790 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [3, 2, 0, 4] 2024-09-15T00:09:30.790 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.3', '3', '2', '0', '4'] 2024-09-15T00:09:30.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.3 3 2 0 4 2024-09-15T00:09:30.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:30.793 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-15T00:09:30.877 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:30.875+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:30.965 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:30.965 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:30.967 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:30.965+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:30.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:30.981 INFO:teuthology.orchestra.run.smithi092.stdout:1577 punch from 0x80d1200 to 0x80dc400, (0xb200 bytes) 2024-09-15T00:09:31.005 INFO:teuthology.orchestra.run.smithi092.stdout:1580 write 0xe5a9e00 thru 0xe5b8855 (0xea56 bytes) 2024-09-15T00:09:31.006 INFO:teuthology.orchestra.run.smithi092.stdout:1582 read 0x5177600 thru 0x517f43c (0x7e3d bytes) 2024-09-15T00:09:31.006 INFO:teuthology.orchestra.run.smithi092.stdout:1583 punch from 0xca3fc00 to 0xca40e00, (0x1200 bytes) 2024-09-15T00:09:31.007 INFO:teuthology.orchestra.run.smithi092.stdout:1585 punch from 0x7abc200 to 0x7abea00, (0x2800 bytes) 2024-09-15T00:09:31.007 INFO:teuthology.orchestra.run.smithi092.stdout:1586 read 0x941f600 thru 0x942483a (0x523b bytes) 2024-09-15T00:09:31.007 INFO:teuthology.orchestra.run.smithi092.stdout:1587 write 0x5a34400 thru 0x5a38a62 (0x4663 bytes) 2024-09-15T00:09:31.008 INFO:teuthology.orchestra.run.smithi092.stdout:1589 punch from 0xc66ae00 to 0xc672600, (0x7800 bytes) 2024-09-15T00:09:31.020 INFO:teuthology.orchestra.run.smithi092.stdout:1590 trunc from 0xe5b8a00 to 0x4ea5c00 2024-09-15T00:09:31.052 INFO:teuthology.orchestra.run.smithi092.stdout:1591 trunc from 0x4ea5c00 to 0xaf41600 2024-09-15T00:09:31.060 INFO:teuthology.orchestra.run.smithi092.stdout:1594 trunc from 0xaf41600 to 0xb23a600 2024-09-15T00:09:31.067 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:31.066+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:31.077 INFO:teuthology.orchestra.run.smithi092.stdout:1595 read 0x7b7ee00 thru 0x7b877ab (0x89ac bytes) 2024-09-15T00:09:31.084 INFO:teuthology.orchestra.run.smithi092.stdout:1596 read 0x4dbc200 thru 0x4dbed91 (0x2b92 bytes) 2024-09-15T00:09:31.084 INFO:teuthology.orchestra.run.smithi092.stdout:1597 read 0x6a5a400 thru 0x6a5f3c9 (0x4fca bytes) 2024-09-15T00:09:31.087 INFO:teuthology.orchestra.run.smithi092.stdout:1601 trunc from 0xb23a600 to 0xd683400 2024-09-15T00:09:31.097 INFO:teuthology.orchestra.run.smithi092.stdout:1602 write 0xa559000 thru 0xa568bb6 (0xfbb7 bytes) 2024-09-15T00:09:31.098 INFO:teuthology.orchestra.run.smithi092.stdout:1604 read 0x55e7000 thru 0x55f0f51 (0x9f52 bytes) 2024-09-15T00:09:31.098 INFO:teuthology.orchestra.run.smithi092.stdout:1607 write 0x843a00 thru 0x8493de (0x59df bytes) 2024-09-15T00:09:31.102 INFO:teuthology.orchestra.run.smithi092.stdout:1608 write 0x68efc00 thru 0x68f5d98 (0x6199 bytes) 2024-09-15T00:09:31.102 INFO:teuthology.orchestra.run.smithi092.stdout:1609 read 0x4fc4600 thru 0x4fce2dc (0x9cdd bytes) 2024-09-15T00:09:31.107 INFO:teuthology.orchestra.run.smithi092.stdout:1612 trunc from 0xd683400 to 0x5c34800 2024-09-15T00:09:31.129 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:31.130 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:31.137 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:31.177 INFO:teuthology.orchestra.run.smithi092.stdout:1613 read 0x8ba800 thru 0x8c3b46 (0x9347 bytes) 2024-09-15T00:09:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:1614 write 0x97f200 thru 0x98b1ab (0xbfac bytes) 2024-09-15T00:09:31.181 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:31.179+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:1616 trunc from 0x5c34800 to 0x680ba00 2024-09-15T00:09:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:1617 write 0x1bb9400 thru 0x1bbb3f2 (0x1ff3 bytes) 2024-09-15T00:09:31.250 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:31.250 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:1619 write 0x8b1d800 thru 0x8b2227a (0x4a7b bytes) 2024-09-15T00:09:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:1620 read 0x68c7600 thru 0x68d2505 (0xaf06 bytes) 2024-09-15T00:09:31.255 INFO:teuthology.orchestra.run.smithi092.stdout:1621 read 0x8a28600 thru 0x8a2bbb9 (0x35ba bytes) 2024-09-15T00:09:31.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:1622 write 0xcd2c800 thru 0xcd3c5ad (0xfdae bytes) 2024-09-15T00:09:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:1624 read 0x6cf8800 thru 0x6d074ef (0xecf0 bytes) 2024-09-15T00:09:31.274 INFO:teuthology.orchestra.run.smithi092.stdout:1625 read 0xf72e00 thru 0xf7cd61 (0x9f62 bytes) 2024-09-15T00:09:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:1628 read 0xbb12c00 thru 0xbb13ff3 (0x13f4 bytes) 2024-09-15T00:09:31.275 INFO:teuthology.orchestra.run.smithi092.stdout:1630 read 0x1a40400 thru 0x1a4c6c7 (0xc2c8 bytes) 2024-09-15T00:09:31.276 INFO:teuthology.orchestra.run.smithi092.stdout:1632 flatten 2024-09-15T00:09:31.292 INFO:teuthology.orchestra.run.smithi092.stdout:1633 trunc from 0xcd3c600 to 0x91f2600 2024-09-15T00:09:31.300 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:31.298+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:31.308 INFO:teuthology.orchestra.run.smithi092.stdout:1635 read 0x1a01400 thru 0x1a0c5ba (0xb1bb bytes) 2024-09-15T00:09:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:1636 write 0x8c24800 thru 0x8c253ed (0xbee bytes) 2024-09-15T00:09:31.313 INFO:teuthology.orchestra.run.smithi092.stdout:1638 punch from 0x3331600 to 0x333e600, (0xd000 bytes) 2024-09-15T00:09:31.323 INFO:teuthology.orchestra.run.smithi092.stdout:1640 write 0xa169800 thru 0xa17485b (0xb05c bytes) 2024-09-15T00:09:31.324 INFO:teuthology.orchestra.run.smithi092.stdout:1643 read 0x19d9a00 thru 0x19e7ac8 (0xe0c9 bytes) 2024-09-15T00:09:31.328 INFO:teuthology.orchestra.run.smithi092.stdout:1647 read 0xdf6e00 thru 0xdff74d (0x894e bytes) 2024-09-15T00:09:31.340 INFO:teuthology.orchestra.run.smithi092.stdout:1653 write 0xa91e200 thru 0xa9251a6 (0x6fa7 bytes) 2024-09-15T00:09:31.354 INFO:teuthology.orchestra.run.smithi092.stdout:1655 write 0xd5f2200 thru 0xd60028a (0xe08b bytes) 2024-09-15T00:09:31.355 INFO:teuthology.orchestra.run.smithi092.stdout:1658 clone 15 order 20 su 524288 sc 2 2024-09-15T00:09:31.370 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:31.368+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:31.408 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:31.408 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:31.416 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:31.500 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:31.498+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:31.564 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:31.564 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:31.571 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:31.570+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:31.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:31.673 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:31.671+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:31.675 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:31.675 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:31.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:31.773 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:31.771+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:31.785 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:31.785 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:31.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:31.874 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:31.873+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:31.902 INFO:teuthology.orchestra.run.smithi027.stderr:set 3.3 pg_upmap_items mapping to [3->2,0->4] 2024-09-15T00:09:31.906 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:31.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:31.973+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:31.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:09:32.049 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:32.049 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:32.057 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:32.076 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:32.074+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:32.162 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:32.163 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:32.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:32.215 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:32.213+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:32.277 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:32.277 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:32.278 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:32.276+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:32.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:32.315 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-15T00:09:32.378 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:32.376+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:32.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:32.479 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:32.477+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:32.579 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:32.578+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:32.627 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:32.627 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:32.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:32.680 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:32.678+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:32.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:32.781 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:32.779+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:32.782 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:32.782 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:32.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:32.826 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:32.826 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:32.843 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:32.844 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:32.860 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:32.877 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:32.877 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:32.883 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:32.881+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:32.892 INFO:teuthology.orchestra.run.smithi092.stdout:1660 read 0xa992e00 thru 0xa99ad64 (0x7f65 bytes) 2024-09-15T00:09:32.893 INFO:teuthology.orchestra.run.smithi092.stdout:1662 punch from 0x3aa8600 to 0x3ab5e00, (0xd800 bytes) 2024-09-15T00:09:32.894 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:32.894 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:32.901 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:32.901 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:09:32.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:32.913 INFO:teuthology.orchestra.run.smithi092.stdout:1664 trunc from 0xd600400 to 0x334a600 2024-09-15T00:09:32.915 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:32.915 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:32.942 INFO:teuthology.orchestra.run.smithi092.stdout:1668 punch from 0x1fcbe00 to 0x1fce600, (0x2800 bytes) 2024-09-15T00:09:32.942 INFO:teuthology.orchestra.run.smithi092.stdout:1669 read 0x138200 thru 0x144151 (0xbf52 bytes) 2024-09-15T00:09:32.954 INFO:teuthology.orchestra.run.smithi092.stdout:1670 trunc from 0x334a600 to 0xc6ed400 2024-09-15T00:09:32.970 INFO:teuthology.orchestra.run.smithi092.stdout:1671 read 0x79ea400 thru 0x79ecd9f (0x29a0 bytes) 2024-09-15T00:09:32.970 INFO:teuthology.orchestra.run.smithi092.stdout:1672 write 0x2033400 thru 0x203f795 (0xc396 bytes) 2024-09-15T00:09:32.970 INFO:teuthology.orchestra.run.smithi092.stdout:1673 read 0x1e99c00 thru 0x1e9f97f (0x5d80 bytes) 2024-09-15T00:09:32.971 INFO:teuthology.orchestra.run.smithi092.stdout:1674 write 0x177c600 thru 0x177c743 (0x144 bytes) 2024-09-15T00:09:32.972 INFO:teuthology.orchestra.run.smithi092.stdout:1675 write 0xb89d800 thru 0xb89fefb (0x26fc bytes) 2024-09-15T00:09:32.972 INFO:teuthology.orchestra.run.smithi092.stdout:1677 write 0x2cd2600 thru 0x2ce0fab (0xe9ac bytes) 2024-09-15T00:09:32.978 INFO:teuthology.orchestra.run.smithi092.stdout:1680 trunc from 0xc6ed400 to 0x7f9a800 2024-09-15T00:09:32.984 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:32.982+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:33.008 INFO:teuthology.orchestra.run.smithi092.stdout:1681 write 0x7367800 thru 0x736af35 (0x3736 bytes) 2024-09-15T00:09:33.008 INFO:teuthology.orchestra.run.smithi092.stdout:1684 read 0x5c91800 thru 0x5c96cdb (0x54dc bytes) 2024-09-15T00:09:33.026 INFO:teuthology.orchestra.run.smithi092.stdout:1685 write 0xc417000 thru 0xc419f45 (0x2f46 bytes) 2024-09-15T00:09:33.032 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:33.032 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:33.033 INFO:teuthology.orchestra.run.smithi092.stdout:1687 read 0xa8ada00 thru 0xa8b8f4a (0xb54b bytes) 2024-09-15T00:09:33.033 INFO:teuthology.orchestra.run.smithi092.stdout:1688 write 0xc3e3a00 thru 0xc3e5020 (0x1621 bytes) 2024-09-15T00:09:33.044 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:33.046 INFO:teuthology.orchestra.run.smithi092.stdout:1689 trunc from 0xc41a000 to 0xc11600 2024-09-15T00:09:33.085 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:33.083+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:33.090 INFO:teuthology.orchestra.run.smithi092.stdout:1690 write 0x77a7600 thru 0x77abd2a (0x472b bytes) 2024-09-15T00:09:33.091 INFO:teuthology.orchestra.run.smithi092.stdout:1691 write 0x3083800 thru 0x30838b8 (0xb9 bytes) 2024-09-15T00:09:33.107 INFO:teuthology.orchestra.run.smithi092.stdout:1692 write 0xa119600 thru 0xa11a6b6 (0x10b7 bytes) 2024-09-15T00:09:33.108 INFO:teuthology.orchestra.run.smithi092.stdout:1693 write 0x1cee400 thru 0x1ceef6c (0xb6d bytes) 2024-09-15T00:09:33.109 INFO:teuthology.orchestra.run.smithi092.stdout:1694 read 0x7313200 thru 0x731f095 (0xbe96 bytes) 2024-09-15T00:09:33.109 INFO:teuthology.orchestra.run.smithi092.stdout:1696 write 0x9f32200 thru 0x9f41d63 (0xfb64 bytes) 2024-09-15T00:09:33.114 INFO:teuthology.orchestra.run.smithi092.stdout:1700 trunc from 0xa11a800 to 0xeb38800 2024-09-15T00:09:33.126 INFO:teuthology.orchestra.run.smithi092.stdout:1701 read 0x315aa00 thru 0x315d1bb (0x27bc bytes) 2024-09-15T00:09:33.127 INFO:teuthology.orchestra.run.smithi092.stdout:1702 read 0x6f73e00 thru 0x6f75015 (0x1216 bytes) 2024-09-15T00:09:33.127 INFO:teuthology.orchestra.run.smithi092.stdout:1704 punch from 0x822fe00 to 0x8232e00, (0x3000 bytes) 2024-09-15T00:09:33.127 INFO:teuthology.orchestra.run.smithi092.stdout:1706 write 0x34ec800 thru 0x34edf52 (0x1753 bytes) 2024-09-15T00:09:33.141 INFO:teuthology.orchestra.run.smithi092.stdout:1710 trunc from 0xeb38800 to 0x478bc00 2024-09-15T00:09:33.156 INFO:teuthology.orchestra.run.smithi092.stdout:1712 write 0x3da4600 thru 0x3dad868 (0x9269 bytes) 2024-09-15T00:09:33.157 INFO:teuthology.orchestra.run.smithi092.stdout:1713 read 0x861200 thru 0x862882 (0x1683 bytes) 2024-09-15T00:09:33.158 INFO:teuthology.orchestra.run.smithi092.stdout:1714 read 0x2565c00 thru 0x25721e4 (0xc5e5 bytes) 2024-09-15T00:09:33.160 INFO:teuthology.orchestra.run.smithi092.stdout:1717 trunc from 0x478bc00 to 0x6469a00 2024-09-15T00:09:33.171 INFO:teuthology.orchestra.run.smithi092.stdout:1719 read 0x5b10600 thru 0x5b14ead (0x48ae bytes) 2024-09-15T00:09:33.172 INFO:teuthology.orchestra.run.smithi092.stdout:1721 write 0x294be00 thru 0x2952d0e (0x6f0f bytes) 2024-09-15T00:09:33.172 INFO:teuthology.orchestra.run.smithi092.stdout:1722 read 0x34d2000 thru 0x34da915 (0x8916 bytes) 2024-09-15T00:09:33.186 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:33.184+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:33.188 INFO:teuthology.orchestra.run.smithi092.stdout:1724 write 0x97f7c00 thru 0x980736e (0xf76f bytes) 2024-09-15T00:09:33.189 INFO:teuthology.orchestra.run.smithi092.stdout:1725 punch from 0x63ffc00 to 0x640de00, (0xe200 bytes) 2024-09-15T00:09:33.190 INFO:teuthology.orchestra.run.smithi092.stdout:1726 read 0x4f0e200 thru 0x4f1da48 (0xf849 bytes) 2024-09-15T00:09:33.190 INFO:teuthology.orchestra.run.smithi092.stdout:1728 write 0x1ee600 thru 0x1fac0b (0xc60c bytes) 2024-09-15T00:09:33.213 INFO:teuthology.orchestra.run.smithi092.stdout:1732 write 0xe9c4a00 thru 0xe9cfbc9 (0xb1ca bytes) 2024-09-15T00:09:33.214 INFO:teuthology.orchestra.run.smithi092.stdout:1733 write 0xc636a00 thru 0xc63f95f (0x8f60 bytes) 2024-09-15T00:09:33.215 INFO:teuthology.orchestra.run.smithi092.stdout:1734 read 0x65d9e00 thru 0x65e3c38 (0x9e39 bytes) 2024-09-15T00:09:33.215 INFO:teuthology.orchestra.run.smithi092.stdout:1736 read 0x271cc00 thru 0x272a7d3 (0xdbd4 bytes) 2024-09-15T00:09:33.215 INFO:teuthology.orchestra.run.smithi092.stdout:1737 write 0xeece00 thru 0xef1744 (0x4945 bytes) 2024-09-15T00:09:33.215 INFO:teuthology.orchestra.run.smithi092.stdout:1738 punch from 0x8036400 to 0x803f000, (0x8c00 bytes) 2024-09-15T00:09:33.215 INFO:teuthology.orchestra.run.smithi092.stdout:1739 write 0xe6a00 thru 0xef805 (0x8e06 bytes) 2024-09-15T00:09:33.217 INFO:teuthology.orchestra.run.smithi092.stdout:1741 write 0xc0c8000 thru 0xc0d7808 (0xf809 bytes) 2024-09-15T00:09:33.217 INFO:teuthology.orchestra.run.smithi092.stdout:1743 punch from 0x1bd3200 to 0x1bd7000, (0x3e00 bytes) 2024-09-15T00:09:33.219 INFO:teuthology.orchestra.run.smithi092.stdout:1744 write 0x4329800 thru 0x432c495 (0x2c96 bytes) 2024-09-15T00:09:33.219 INFO:teuthology.orchestra.run.smithi092.stdout:1745 write 0x9f93600 thru 0x9f9e8e1 (0xb2e2 bytes) 2024-09-15T00:09:33.219 INFO:teuthology.orchestra.run.smithi092.stdout:1746 read 0x8367200 thru 0x83700ac (0x8ead bytes) 2024-09-15T00:09:33.219 INFO:teuthology.orchestra.run.smithi092.stdout:1748 punch from 0xa9c9200 to 0xa9cac00, (0x1a00 bytes) 2024-09-15T00:09:33.219 INFO:teuthology.orchestra.run.smithi092.stdout:1750 write 0xcb7e800 thru 0xcb86d85 (0x8586 bytes) 2024-09-15T00:09:33.219 INFO:teuthology.orchestra.run.smithi092.stdout:1751 read 0x1997200 thru 0x199e519 (0x731a bytes) 2024-09-15T00:09:33.219 INFO:teuthology.orchestra.run.smithi092.stdout:1752 read 0x2126800 thru 0x2133c81 (0xd482 bytes) 2024-09-15T00:09:33.220 INFO:teuthology.orchestra.run.smithi092.stdout:1753 punch from 0x557be00 to 0x5580600, (0x4800 bytes) 2024-09-15T00:09:33.220 INFO:teuthology.orchestra.run.smithi092.stdout:1756 punch from 0x2e04600 to 0x2e0d000, (0x8a00 bytes) 2024-09-15T00:09:33.220 INFO:teuthology.orchestra.run.smithi092.stdout:1757 write 0x14cac00 thru 0x14cdb02 (0x2f03 bytes) 2024-09-15T00:09:33.220 INFO:teuthology.orchestra.run.smithi092.stdout:1758 write 0x1f41000 thru 0x1f46d80 (0x5d81 bytes) 2024-09-15T00:09:33.220 INFO:teuthology.orchestra.run.smithi092.stdout:1759 read 0x4c53400 thru 0x4c5ba56 (0x8657 bytes) 2024-09-15T00:09:33.220 INFO:teuthology.orchestra.run.smithi092.stdout:1760 read 0x6105000 thru 0x6109f9c (0x4f9d bytes) 2024-09-15T00:09:33.220 INFO:teuthology.orchestra.run.smithi092.stdout:1763 read 0xc726200 thru 0xc72afb4 (0x4db5 bytes) 2024-09-15T00:09:33.221 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8832 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.063773187000001, 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056083069999999999, 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8832 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2099999999999998e-07 2024-09-15T00:09:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153008+0000", 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.187e-06 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153012+0000", 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8909999999999997e-06 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153015+0000", 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8559999999999998e-06 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202369+0000", 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.049354393000000003 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202431+0000", 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2235000000000003e-05 2024-09-15T00:09:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202749+0000", 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00031714299999999998 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204050+0000", 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00130164 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209033+0000", 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049826590000000004 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209063+0000", 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0176999999999999e-05 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209089+0000", 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5568e-05 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8833 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.063758708, 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056156170999999998, 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8833 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1299999999999999e-07 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153022+0000", 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5829999999999999e-06 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153025+0000", 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.022e-06 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153029+0000", 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8500000000000004e-06 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203131+0000", 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.05010212 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203216+0000", 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.543e-05 2024-09-15T00:09:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203493+0000", 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027699000000000002 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206606+0000", 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003112315 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209120+0000", 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002514863 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209149+0000", 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8685000000000001e-05 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209176+0000", 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7100000000000001e-05 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8834 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.063747166000001, 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056218377999999999, 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8834 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.16e-07 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153034+0000", 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.844e-06 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153039+0000", 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8369999999999996e-06 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153041+0000", 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5620000000000002e-06 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204094+0000", 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051053378000000003 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204144+0000", 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9907999999999997e-05 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204342+0000", 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000197596 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206613+0000", 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022710579999999998 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209204+0000", 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0025907780000000002 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209230+0000", 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6041999999999999e-05 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209250+0000", 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0159000000000001e-05 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8835 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.063734756000001, 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060077884999999998, 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8835 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.16e-07 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153046+0000", 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153049+0000", 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153051+0000", 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.52e-06 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204715+0000", 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051664032999999998 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204804+0000", 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8383999999999999e-05 2024-09-15T00:09:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205194+0000", 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039034399999999999 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206890+0000", 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016954680000000001 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213068+0000", 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006178411 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213097+0000", 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9433000000000001e-05 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213122+0000", 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4553999999999999e-05 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8836 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.063724370999999, 2024-09-15T00:09:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060148357, 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8836 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153055+0000", 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0100000000000001e-07 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153056+0000", 2024-09-15T00:09:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6220000000000001e-06 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153060+0000", 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5499999999999999e-06 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153062+0000", 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6510000000000001e-06 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205724+0000", 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.052661396999999999 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205802+0000", 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8668e-05 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206109+0000", 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030663400000000002 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.208482+0000", 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023725019999999999 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213151+0000", 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046689660000000001 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213181+0000", 2024-09-15T00:09:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0324999999999998e-05 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213203+0000", 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1841e-05 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8837 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.063713498, 2024-09-15T00:09:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060386983999999998, 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8837 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1500000000000001e-07 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153067+0000", 2024-09-15T00:09:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.764e-06 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153070+0000", 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.869e-06 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153073+0000", 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.548e-06 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206907+0000", 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053834290999999999 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206953+0000", 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5648000000000001e-05 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207282+0000", 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032939999999999998 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209097+0000", 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018149170000000001 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213388+0000", 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042910989999999996 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213430+0000", 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2465000000000001e-05 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213452+0000", 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1767999999999999e-05 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8838 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.063703256, 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060498379999999997, 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8838 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.04e-07 2024-09-15T00:09:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153077+0000", 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.389e-06 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153080+0000", 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8499999999999998e-06 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153083+0000", 2024-09-15T00:09:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.565e-06 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207872+0000", 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.054789110000000002 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207926+0000", 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4515000000000002e-05 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.208143+0000", 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000216784 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209511+0000", 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013684350000000001 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213494+0000", 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039824889999999996 2024-09-15T00:09:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213535+0000", 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0865999999999998e-05 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213574+0000", 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9172999999999999e-05 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3358 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7684096~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1706088729999999, 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055739138000000001, 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3358 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046177+0000", 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7000000000000002e-06 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046184+0000", 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.029e-06 2024-09-15T00:09:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046189+0000", 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9459999999999997e-06 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101686+0000", 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055497367999999998 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101778+0000", 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2077000000000006e-05 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101909+0000", 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013064700000000001 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.271 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3359 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7733248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1705811289999999, 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055953638999999999, 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3359 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:33.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046200+0000", 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046207+0000", 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8050000000000001e-06 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046213+0000", 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3489999999999999e-06 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101957+0000", 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055744316000000002 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102048+0000", 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1464000000000004e-05 2024-09-15T00:09:33.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102151+0000", 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102874 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3360 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7782400~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1705587180000001, 2024-09-15T00:09:33.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056128863000000001, 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3360 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046223+0000", 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046229+0000", 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6839999999999999e-06 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046234+0000", 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6800000000000001e-06 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102187+0000", 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.05595282 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102262+0000", 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5371999999999996e-05 2024-09-15T00:09:33.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102349+0000", 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6706999999999995e-05 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3361 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7831552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1705378839999998, 2024-09-15T00:09:33.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056322561, 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3361 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046244+0000", 2024-09-15T00:09:33.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.43e-06 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046250+0000", 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5420000000000002e-06 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046255+0000", 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.707e-06 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102382+0000", 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056126796999999999 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102469+0000", 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7023999999999999e-05 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102563+0000", 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4678000000000004e-05 2024-09-15T00:09:33.285 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3362 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7880704~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046262+0000", 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1705168420000001, 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056530960999999998, 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.286 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3362 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046262+0000", 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046262+0000", 2024-09-15T00:09:33.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046492+0000", 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023003100000000001 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046496+0000", 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9210000000000002e-06 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046506+0000", 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0001e-05 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046513+0000", 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9229999999999996e-06 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102610+0000", 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056097039000000001 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102698+0000", 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7860000000000002e-05 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102793+0000", 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5186000000000006e-05 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3363 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7929856~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046548+0000", 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1702306070000001, 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056535131000000002, 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3363 2024-09-15T00:09:33.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046548+0000", 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046548+0000", 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046549+0000", 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6000000000000003e-07 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046552+0000", 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046564+0000", 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1474999999999999e-05 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046572+0000", 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4009999999999997e-06 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102824+0000", 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056251511999999997 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102922+0000", 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8499999999999995e-05 2024-09-15T00:09:33.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103083+0000", 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016119300000000001 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3364 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7979008~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046606+0000", 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.17017236, 2024-09-15T00:09:33.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056797145, 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3364 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046606+0000", 2024-09-15T00:09:33.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046606+0000", 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047399+0000", 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00079283299999999995 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047403+0000", 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1670000000000001e-06 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047413+0000", 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4220000000000006e-06 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047419+0000", 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4450000000000001e-06 2024-09-15T00:09:33.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103137+0000", 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055717322999999999 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103252+0000", 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115701 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103404+0000", 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000151254 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3365 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8028160~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.047431+0000", 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1693477539999999, 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056539155000000001, 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3365 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047431+0000", 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047431+0000", 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047432+0000", 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047434+0000", 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:09:33.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047441+0000", 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.849e-06 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047447+0000", 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.609e-06 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103694+0000", 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056247765999999998 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103808+0000", 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011345699999999999 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103970+0000", 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000162421 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.304 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3366 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8077312~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.047453+0000", 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.169325309, 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056741575000000002, 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.305 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3366 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047453+0000", 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047453+0000", 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047454+0000", 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047456+0000", 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.125e-06 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047462+0000", 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5999999999999997e-06 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047466+0000", 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5990000000000004e-06 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104001+0000", 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056535215 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104093+0000", 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1496999999999995e-05 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104195+0000", 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010213900000000001 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.309 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3367 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8126464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.047473+0000", 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1693060339999999, 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056984806999999998, 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3367 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047473+0000", 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047473+0000", 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049398+0000", 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001925689 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049403+0000", 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7700000000000001e-06 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049415+0000", 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1657999999999999e-05 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049422+0000", 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6350000000000006e-06 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104236+0000", 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.054813957000000003 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104352+0000", 2024-09-15T00:09:33.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011547100000000001 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104458+0000", 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105627 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3370 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8273920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:33.314 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1672600000000002, 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055967186000000002, 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3370 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9999999999999997e-07 2024-09-15T00:09:33.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049522+0000", 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5900000000000002e-06 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049531+0000", 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0270000000000001e-06 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049537+0000", 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8610000000000003e-06 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105279+0000", 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055742236000000001 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105364+0000", 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4993000000000004e-05 2024-09-15T00:09:33.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105486+0000", 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3371 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8323072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.049545+0000", 2024-09-15T00:09:33.319 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1672334630000001, 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056146330000000001, 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3371 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049545+0000", 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049545+0000", 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049546+0000", 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7300000000000001e-07 2024-09-15T00:09:33.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049548+0000", 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5770000000000001e-06 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049740+0000", 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019119300000000001 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049749+0000", 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3850000000000008e-06 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105519+0000", 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055770239999999999 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105601+0000", 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1460999999999998e-05 2024-09-15T00:09:33.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105692+0000", 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1001000000000005e-05 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3372 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8372224~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1670140980000001, 2024-09-15T00:09:33.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056196995, 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3372 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:33.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9100000000000004e-07 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049769+0000", 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8399999999999997e-06 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049777+0000", 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049783+0000", 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:09:33.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105753+0000", 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055970585000000003 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105831+0000", 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7171999999999999e-05 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105962+0000", 2024-09-15T00:09:33.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013104800000000001 2024-09-15T00:09:33.329 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.329 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.329 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.329 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.329 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.329 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:33.330 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:33.328+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:33.332 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:33.332 INFO:teuthology.orchestra.run.smithi092.stdout:1764 write 0x6ec6a00 thru 0x6ed4c45 (0xe246 bytes) 2024-09-15T00:09:33.332 INFO:teuthology.orchestra.run.smithi092.stdout:1765 trunc from 0xe9cfc00 to 0xcf83200 2024-09-15T00:09:33.332 INFO:teuthology.orchestra.run.smithi092.stdout:1770 trunc from 0xcf83200 to 0x1695200 2024-09-15T00:09:33.332 INFO:teuthology.orchestra.run.smithi092.stdout:1772 read 0x5f6a00 thru 0x5f9ac3 (0x30c4 bytes) 2024-09-15T00:09:33.332 INFO:teuthology.orchestra.run.smithi092.stdout:1774 trunc from 0x1695200 to 0x6ede200 2024-09-15T00:09:33.333 INFO:teuthology.orchestra.run.smithi092.stdout:1775 trunc from 0x6ede200 to 0x8976e00 2024-09-15T00:09:33.333 INFO:teuthology.orchestra.run.smithi092.stdout:1779 punch from 0x5dbce00 to 0x5dc4600, (0x7800 bytes) 2024-09-15T00:09:33.333 INFO:teuthology.orchestra.run.smithi092.stdout:1780 trunc from 0x8976e00 to 0xbcff800 2024-09-15T00:09:33.431 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:33.429+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:33.479 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:33.480 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:33.480 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:33.480 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:33.488 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:33.533 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:33.531+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:33.574 INFO:teuthology.orchestra.run.smithi092.stdout:1781 read 0xbb84e00 thru 0xbb88e60 (0x4061 bytes) 2024-09-15T00:09:33.574 INFO:teuthology.orchestra.run.smithi092.stdout:1784 read 0x490c800 thru 0x4918599 (0xbd9a bytes) 2024-09-15T00:09:33.574 INFO:teuthology.orchestra.run.smithi092.stdout:1788 read 0x3626c00 thru 0x362ab9c (0x3f9d bytes) 2024-09-15T00:09:33.574 INFO:teuthology.orchestra.run.smithi092.stdout:1789 write 0x6d00800 thru 0x6d07d5b (0x755c bytes) 2024-09-15T00:09:33.574 INFO:teuthology.orchestra.run.smithi092.stdout:1790 read 0x76af000 thru 0x76b2a78 (0x3a79 bytes) 2024-09-15T00:09:33.574 INFO:teuthology.orchestra.run.smithi092.stdout:1791 punch from 0xa383e00 to 0xa38c400, (0x8600 bytes) 2024-09-15T00:09:33.574 INFO:teuthology.orchestra.run.smithi092.stdout:1792 read 0x2cc7c00 thru 0x2cd30e8 (0xb4e9 bytes) 2024-09-15T00:09:33.574 INFO:teuthology.orchestra.run.smithi092.stdout:1793 punch from 0x7331200 to 0x7332800, (0x1600 bytes) 2024-09-15T00:09:33.574 INFO:teuthology.orchestra.run.smithi092.stdout:1794 punch from 0x37f2800 to 0x37fba00, (0x9200 bytes) 2024-09-15T00:09:33.575 INFO:teuthology.orchestra.run.smithi092.stdout:1795 trunc from 0xbcff800 to 0xc263e00 2024-09-15T00:09:33.587 INFO:teuthology.orchestra.run.smithi092.stdout:1800 write 0x6a7c400 thru 0x6a82e64 (0x6a65 bytes) 2024-09-15T00:09:33.589 INFO:teuthology.orchestra.run.smithi092.stdout:1801 clone 16 order 18 su 131072 sc 9 2024-09-15T00:09:33.633 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:33.632+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:33.639 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:33.639 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:33.640 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:33.640 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:33.640 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:33.647 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:33.734 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:33.733+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:33.755 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7258260569999999, 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:09:33.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.760 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.761 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.761 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:33.761 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.725805802, 2024-09-15T00:09:33.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:09:33.761 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:09:33.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.765 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.725786056, 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.770 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.725766106, 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7257386060000002, 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.776 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:09:33.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.780 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7257217649999999, 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:09:33.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:09:33.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.725705767, 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.785 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:09:33.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:09:33.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.725682779, 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:09:33.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7256568950000002, 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:09:33.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.799 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7256404679999999, 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:33.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:09:33.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.725626197, 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:09:33.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.72561688, 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.809 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.813 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7239478679999998, 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:09:33.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:09:33.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7239128049999999, 2024-09-15T00:09:33.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:33.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:09:33.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7238877779999999, 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.827 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.72386347, 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.828 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:09:33.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:09:33.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2674 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3768320~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.721717124, 2024-09-15T00:09:33.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059850591000000002, 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2674 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031671+0000", 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031682+0000", 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1248e-05 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031693+0000", 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1395e-05 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031705+0000", 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2106000000000001e-05 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091352+0000", 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059646421999999998 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091441+0000", 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9174999999999999e-05 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091521+0000", 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9636999999999997e-05 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.836 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7216712190000001, 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.837 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:09:33.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.841 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.721647087, 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7216177149999998, 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:09:33.846 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:09:33.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:33.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.850 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:33.851 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:33.851 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:33.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:33.853 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:33.851+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:33.953 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:33.952+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:33.966 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:33.967 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:33.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:33.976 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone15 from 0xc263e00 (overlap 0xc11600) to 0x4d3c00 2024-09-15T00:09:34.054 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:34.052+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:34.090 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:34.090 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:34.090 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:34.090 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:34.091 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:34.099 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:34.155 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:34.154+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:34.250 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:34.250 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:34.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:34.250 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.222158533, 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.251 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:09:34.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:34.255 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2221366150000001, 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:34.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:09:34.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:09:34.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2221153429999996, 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.260 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:09:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2203740830000003, 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:09:34.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2203521769999996, 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:09:34.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2203300849999996, 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.275 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:09:34.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2202983459999999, 2024-09-15T00:09:34.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:09:34.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.220000143, 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.284 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:09:34.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:34.288 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2199664380000002, 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:09:34.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2156643650000003, 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:09:34.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2153394950000003, 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.298 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:09:34.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.302 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.21492909, 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:34.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.306 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2148741870000004, 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.307 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:09:34.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2148421520000001, 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:09:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2133643220000003, 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.316 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2133426070000004, 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.321 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:34.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2133211279999996, 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:09:34.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.329 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.211308024, 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:09:34.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:09:34.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2112883830000003, 2024-09-15T00:09:34.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.211241835, 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:34.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:09:34.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:09:34.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:34.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:34.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:09:34.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:09:34.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.343 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:34.343 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:34.343 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:34.344 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-15T00:09:34.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:34.345 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:34.343+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:34.445 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:34.443+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:34.452 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:34.452 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:34.452 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:34.452 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:34.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:34.547 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:34.546+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:34.610 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:34.610 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:34.610 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:34.610 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:34.610 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:34.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:34.660 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:34.658+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:34.749 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:34.747+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:34.762 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4070 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408975434, 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02702953, 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4070 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350999+0000", 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351003+0000", 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351007+0000", 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6820000000000001e-06 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351010+0000", 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7269999999999999e-06 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377929+0000", 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026919887999999999 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377969+0000", 2024-09-15T00:09:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9909e-05 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378028+0000", 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8578000000000001e-05 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4071 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408960554, 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027136008, 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4071 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351014+0000", 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:09:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351015+0000", 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7039999999999999e-06 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351020+0000", 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351023+0000", 2024-09-15T00:09:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378053+0000", 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027029610999999999 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378094+0000", 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1826999999999998e-05 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378149+0000", 2024-09-15T00:09:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4815000000000003e-05 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4072 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408935874999999, 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027223404999999999, 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4072 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351040+0000", 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351046+0000", 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351051+0000", 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378171+0000", 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027120590999999999 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378209+0000", 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7308999999999998e-05 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378261+0000", 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2753999999999997e-05 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4074 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408916980000001, 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027334509999999999, 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4074 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9200000000000002e-07 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351059+0000", 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.012e-06 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351065+0000", 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2129999999999996e-06 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351070+0000", 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8470000000000003e-06 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378285+0000", 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027214449000000002 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378334+0000", 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378391+0000", 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7046000000000002e-05 2024-09-15T00:09:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4075 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408898109000001, 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027546923000000001, 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4075 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351078+0000", 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.905e-06 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351085+0000", 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:09:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351090+0000", 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378423+0000", 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027333051000000001 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378523+0000", 2024-09-15T00:09:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9959999999999998e-05 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378623+0000", 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9962000000000003e-05 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4077 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408875425, 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027660619000000001, 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4077 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351099+0000", 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351101+0000", 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351107+0000", 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7980000000000002e-06 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351111+0000", 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378657+0000", 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027545463999999999 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378702+0000", 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5809999999999997e-05 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378759+0000", 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-15T00:09:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4078 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408847424999999, 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027808405000000001, 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4078 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351127+0000", 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351129+0000", 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351135+0000", 2024-09-15T00:09:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3820000000000001e-06 2024-09-15T00:09:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351141+0000", 2024-09-15T00:09:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2259999999999996e-06 2024-09-15T00:09:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378785+0000", 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027644592999999999 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378855+0000", 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0276000000000003e-05 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378935+0000", 2024-09-15T00:09:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9361000000000001e-05 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4079 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408825788, 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027913778, 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4079 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:09:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351150+0000", 2024-09-15T00:09:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9010000000000001e-06 2024-09-15T00:09:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351157+0000", 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351162+0000", 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378958+0000", 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027795786999999999 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379004+0000", 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6124000000000001e-05 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379062+0000", 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7630999999999998e-05 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4081 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408802107, 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016658, 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4081 2024-09-15T00:09:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351174+0000", 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351181+0000", 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9070000000000001e-06 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351186+0000", 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.651e-06 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379085+0000", 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027898757999999999 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379130+0000", 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5126000000000001e-05 2024-09-15T00:09:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379188+0000", 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8548999999999998e-05 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4082 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408782052999999, 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028118457999999999, 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4082 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351194+0000", 2024-09-15T00:09:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7519999999999999e-06 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351200+0000", 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.801e-06 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351205+0000", 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379213+0000", 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028007539000000001 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379254+0000", 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1817999999999998e-05 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379310+0000", 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5850000000000002e-05 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4083 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408760333, 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028218838999999999, 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4083 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351214+0000", 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351216+0000", 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351222+0000", 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-15T00:09:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351226+0000", 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379333+0000", 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028106740000000002 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379376+0000", 2024-09-15T00:09:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3581000000000002e-05 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379432+0000", 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5868000000000001e-05 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4085 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408731829000001, 2024-09-15T00:09:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311823, 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4085 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0699999999999998e-07 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351244+0000", 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.782e-06 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351248+0000", 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6289999999999999e-06 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351250+0000", 2024-09-15T00:09:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379457+0000", 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028206479 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379499+0000", 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1637000000000001e-05 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379554+0000", 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5248999999999999e-05 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4086 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408719988, 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02842368, 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4086 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351259+0000", 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9520000000000002e-06 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351262+0000", 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351265+0000", 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379576+0000", 2024-09-15T00:09:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311228000000001 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379619+0000", 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2595000000000001e-05 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379678+0000", 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4087 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408705633, 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523903999999999, 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.824 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4087 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351269+0000", 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351270+0000", 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1090000000000001e-06 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351273+0000", 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9189999999999999e-06 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351275+0000", 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379701+0000", 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028425813000000001 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379742+0000", 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0769000000000002e-05 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379792+0000", 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4088 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408689498999999, 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028621284, 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4088 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.829 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351285+0000", 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351286+0000", 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.186e-06 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351289+0000", 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351292+0000", 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379815+0000", 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523211 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379857+0000", 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1807999999999997e-05 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379906+0000", 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8594999999999998e-05 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.833 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4090 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408678414000001, 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028748587999999999, 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4090 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351296+0000", 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351297+0000", 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1060000000000001e-06 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351300+0000", 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9110000000000002e-06 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351302+0000", 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379947+0000", 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028645177000000001 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379991+0000", 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3680999999999997e-05 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380044+0000", 2024-09-15T00:09:34.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2957000000000001e-05 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4091 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408668859, 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028864198000000001, 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4091 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351306+0000", 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351309+0000", 2024-09-15T00:09:34.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351312+0000", 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380069+0000", 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028757933999999999 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380113+0000", 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3810000000000002e-05 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380169+0000", 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5887999999999998e-05 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.842 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4092 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408659501000001, 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028973819000000001, 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4092 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351315+0000", 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9999999999999999e-07 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351316+0000", 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.95e-07 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351318+0000", 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8629999999999999e-06 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351321+0000", 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380195+0000", 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028873868 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380238+0000", 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2883999999999998e-05 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380288+0000", 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0096000000000003e-05 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4094 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.847 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408649882000001, 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02907964, 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4094 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351325+0000", 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.054e-06 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351328+0000", 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351331+0000", 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4679999999999999e-06 2024-09-15T00:09:34.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380312+0000", 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028981587999999999 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380355+0000", 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2688000000000002e-05 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380404+0000", 2024-09-15T00:09:34.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8767999999999998e-05 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4095 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.408640495, 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029185068000000002, 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:34.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4095 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:34.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351334+0000", 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.04e-07 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351335+0000", 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0300000000000001e-06 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351339+0000", 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351344+0000", 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0560000000000004e-06 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380428+0000", 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02908444 2024-09-15T00:09:34.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380470+0000", 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1338000000000003e-05 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380518+0000", 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8724999999999999e-05 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:34.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:34.857 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:34.858 INFO:teuthology.orchestra.run.smithi092.stdout:1804 read 0xb914400 thru 0xb91d374 (0x8f75 bytes) 2024-09-15T00:09:34.858 INFO:teuthology.orchestra.run.smithi092.stdout:1805 punch from 0x942a00 to 0x946c00, (0x4200 bytes) 2024-09-15T00:09:34.858 INFO:teuthology.orchestra.run.smithi092.stdout:1806 write 0x6c35e00 thru 0x6c3db31 (0x7d32 bytes) 2024-09-15T00:09:34.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:34.859 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:34.856+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:34.870 INFO:teuthology.orchestra.run.smithi092.stdout:1807 write 0xc512c00 thru 0xc51cc52 (0xa053 bytes) 2024-09-15T00:09:34.870 INFO:teuthology.orchestra.run.smithi092.stdout:1809 write 0x8a69e00 thru 0x8a6fa61 (0x5c62 bytes) 2024-09-15T00:09:34.880 INFO:teuthology.orchestra.run.smithi092.stdout:1812 trunc from 0xc51ce00 to 0x5a10e00 2024-09-15T00:09:34.920 INFO:teuthology.orchestra.run.smithi092.stdout:1814 trunc from 0x5a10e00 to 0xec78200 2024-09-15T00:09:34.920 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:09:34.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:34.936 INFO:teuthology.orchestra.run.smithi092.stdout:1815 punch from 0x8178c00 to 0x8186800, (0xdc00 bytes) 2024-09-15T00:09:34.936 INFO:teuthology.orchestra.run.smithi092.stdout:1817 read 0xa2cbe00 thru 0xa2d7238 (0xb439 bytes) 2024-09-15T00:09:34.936 INFO:teuthology.orchestra.run.smithi092.stdout:1819 write 0x5a3e800 thru 0x5a46280 (0x7a81 bytes) 2024-09-15T00:09:34.937 INFO:teuthology.orchestra.run.smithi092.stdout:1821 punch from 0xc45c000 to 0xc463c00, (0x7c00 bytes) 2024-09-15T00:09:34.937 INFO:teuthology.orchestra.run.smithi092.stdout:1822 write 0x59a4c00 thru 0x59a91e5 (0x45e6 bytes) 2024-09-15T00:09:34.937 INFO:teuthology.orchestra.run.smithi092.stdout:1826 read 0xe8b0800 thru 0xe8bb339 (0xab3a bytes) 2024-09-15T00:09:34.937 INFO:teuthology.orchestra.run.smithi092.stdout:1828 write 0x5775a00 thru 0x57829b5 (0xcfb6 bytes) 2024-09-15T00:09:34.947 INFO:teuthology.orchestra.run.smithi092.stdout:1831 trunc from 0xec78200 to 0x5d6a400 2024-09-15T00:09:34.964 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:34.962+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:34.973 INFO:teuthology.orchestra.run.smithi092.stdout:1836 punch from 0x2c4fc00 to 0x2c51600, (0x1a00 bytes) 2024-09-15T00:09:34.975 INFO:teuthology.orchestra.run.smithi092.stdout:1837 write 0x762c00 thru 0x7678a5 (0x4ca6 bytes) 2024-09-15T00:09:34.976 INFO:teuthology.orchestra.run.smithi092.stdout:1840 write 0x1fb9400 thru 0x1fc5cff (0xc900 bytes) 2024-09-15T00:09:34.976 INFO:teuthology.orchestra.run.smithi092.stdout:1842 read 0x56c400 thru 0x56c62d (0x22e bytes) 2024-09-15T00:09:34.976 INFO:teuthology.orchestra.run.smithi092.stdout:1843 write 0x2208000 thru 0x22176a1 (0xf6a2 bytes) 2024-09-15T00:09:34.976 INFO:teuthology.orchestra.run.smithi092.stdout:1845 write 0x4940400 thru 0x4940646 (0x247 bytes) 2024-09-15T00:09:34.977 INFO:teuthology.orchestra.run.smithi092.stdout:1846 trunc from 0x5d6a400 to 0x64ffc00 2024-09-15T00:09:35.009 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:35.010 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:35.010 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:35.019 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:35.020 INFO:teuthology.orchestra.run.smithi092.stdout:1847 write 0xd9f4000 thru 0xd9ff8d3 (0xb8d4 bytes) 2024-09-15T00:09:35.020 INFO:teuthology.orchestra.run.smithi092.stdout:1848 read 0x6b7600 thru 0x6c5750 (0xe151 bytes) 2024-09-15T00:09:35.021 INFO:teuthology.orchestra.run.smithi092.stdout:1849 punch from 0x57bd200 to 0x57bea00, (0x1800 bytes) 2024-09-15T00:09:35.021 INFO:teuthology.orchestra.run.smithi092.stdout:1850 read 0x733c600 thru 0x734bfd3 (0xf9d4 bytes) 2024-09-15T00:09:35.021 INFO:teuthology.orchestra.run.smithi092.stdout:1851 write 0xb07a00 thru 0xb12543 (0xab44 bytes) 2024-09-15T00:09:35.022 INFO:teuthology.orchestra.run.smithi092.stdout:1854 read 0x844d200 thru 0x8452a6d (0x586e bytes) 2024-09-15T00:09:35.022 INFO:teuthology.orchestra.run.smithi092.stdout:1855 punch from 0x796e00 to 0x79bc00, (0x4e00 bytes) 2024-09-15T00:09:35.022 INFO:teuthology.orchestra.run.smithi092.stdout:1856 punch from 0x6baca00 to 0x6bade00, (0x1400 bytes) 2024-09-15T00:09:35.026 INFO:teuthology.orchestra.run.smithi092.stdout:1857 trunc from 0xd9ffa00 to 0x9e35400 2024-09-15T00:09:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:1859 write 0x385a000 thru 0x3868002 (0xe003 bytes) 2024-09-15T00:09:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:1861 read 0x5cd3e00 thru 0x5ce2d23 (0xef24 bytes) 2024-09-15T00:09:35.062 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:35.061+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:35.075 INFO:teuthology.orchestra.run.smithi092.stdout:1862 write 0xb8a1800 thru 0xb8ab019 (0x981a bytes) 2024-09-15T00:09:35.075 INFO:teuthology.orchestra.run.smithi092.stdout:1866 read 0x3d3a200 thru 0x3d3ab52 (0x953 bytes) 2024-09-15T00:09:35.075 INFO:teuthology.orchestra.run.smithi092.stdout:1869 punch from 0x5de8e00 to 0x5dee000, (0x5200 bytes) 2024-09-15T00:09:35.075 INFO:teuthology.orchestra.run.smithi092.stdout:1871 write 0x2932c00 thru 0x293fe0e (0xd20f bytes) 2024-09-15T00:09:35.080 INFO:teuthology.orchestra.run.smithi092.stdout:1873 trunc from 0xb8ab200 to 0x70b8200 2024-09-15T00:09:35.098 INFO:teuthology.orchestra.run.smithi092.stdout:1874 punch from 0x5e2c200 to 0x5e35600, (0x9400 bytes) 2024-09-15T00:09:35.103 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:35.115 INFO:teuthology.orchestra.run.smithi092.stdout:1876 trunc from 0x70b8200 to 0xe7b1a00 2024-09-15T00:09:35.123 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:35.124 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:35.124 INFO:teuthology.orchestra.run.smithi092.stdout:1877 punch from 0x31bf600 to 0x31c9a00, (0xa400 bytes) 2024-09-15T00:09:35.132 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:35.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:35.132 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:35.132 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:35.132 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:35.137 INFO:teuthology.orchestra.run.smithi092.stdout:1879 trunc from 0xe7b1a00 to 0x4c7fc00 2024-09-15T00:09:35.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:35.145 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:35.145 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:35.157 INFO:teuthology.orchestra.run.smithi092.stdout:1881 read 0x2d02800 thru 0x2d0fdf8 (0xd5f9 bytes) 2024-09-15T00:09:35.159 INFO:teuthology.orchestra.run.smithi092.stdout:1883 clone 17 order 25 su 262144 sc 2 2024-09-15T00:09:35.163 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:35.163 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:35.181 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:35.181 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:35.182 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:35.180+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:35.197 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:35.261 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:35.259+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:35.283 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:35.283 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:35.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:35.362 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:35.361+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:35.394 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:35.395 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:35.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:35.463 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:35.461+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:35.545 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:35.546 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:35.554 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:35.564 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:35.562+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:35.658 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:35.658 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:35.665 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:35.663+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:35.666 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:35.765 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:35.763+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:35.778 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:35.778 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:35.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:35.870 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:35.868+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:35.926 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:09:35.937 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:35.937 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:35.946 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:35.967 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:35.965+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:36.007 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone16 2024-09-15T00:09:36.068 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:36.066+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:36.104 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:36.105 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:36.112 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:36.196 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:36.194+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:36.261 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:36.261 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:36.270 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:36.270 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:36.268+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:36.370 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:36.369+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:36.382 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-15T00:09:36.421 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:36.421 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:36.430 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:36.465 INFO:teuthology.orchestra.run.smithi092.stdout:1886 read 0x4581a00 thru 0x458ce69 (0xb46a bytes) 2024-09-15T00:09:36.465 INFO:teuthology.orchestra.run.smithi092.stdout:1888 write 0x4108e00 thru 0x41148ab (0xbaac bytes) 2024-09-15T00:09:36.471 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:36.470+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:36.490 INFO:teuthology.orchestra.run.smithi092.stdout:1889 write 0xaa03a00 thru 0xaa12e1a (0xf41b bytes) 2024-09-15T00:09:36.497 INFO:teuthology.orchestra.run.smithi092.stdout:1890 write 0x25d3800 thru 0x25d6d67 (0x3568 bytes) 2024-09-15T00:09:36.498 INFO:teuthology.orchestra.run.smithi092.stdout:1891 trunc from 0xaa13000 to 0xb3cf200 2024-09-15T00:09:36.517 INFO:teuthology.orchestra.run.smithi092.stdout:1892 read 0x68d8c00 thru 0x68e15b4 (0x89b5 bytes) 2024-09-15T00:09:36.517 INFO:teuthology.orchestra.run.smithi092.stdout:1893 read 0x9d9e200 thru 0x9da0148 (0x1f49 bytes) 2024-09-15T00:09:36.518 INFO:teuthology.orchestra.run.smithi092.stdout:1896 write 0x620e000 thru 0x62118c9 (0x38ca bytes) 2024-09-15T00:09:36.530 INFO:teuthology.orchestra.run.smithi092.stdout:1897 write 0xc722a00 thru 0xc7254aa (0x2aab bytes) 2024-09-15T00:09:36.531 INFO:teuthology.orchestra.run.smithi092.stdout:1900 read 0x19afc00 thru 0x19b2485 (0x2886 bytes) 2024-09-15T00:09:36.531 INFO:teuthology.orchestra.run.smithi092.stdout:1901 read 0xc347600 thru 0xc353c24 (0xc625 bytes) 2024-09-15T00:09:36.531 INFO:teuthology.orchestra.run.smithi092.stdout:1902 clone 18 order 20 su 8192 sc 3 2024-09-15T00:09:36.545 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:36.545 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:36.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:36.572 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:36.571+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:36.661 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:36.662 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:36.670 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:36.673 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:36.671+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:36.773 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:36.772+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:36.776 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:36.776 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:36.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:36.873 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:36.872+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:36.892 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:36.892 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:36.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:36.920 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-15T00:09:36.920 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:09:36.920 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T00:09:36.921 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:09:36.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:36.972+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:37.001 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone17 from 0xc725600 (overlap 0x4c7fc00) to 0x2de6400 2024-09-15T00:09:37.010 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:37.010 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:37.021 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:37.075 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:37.073+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:37.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:37.175 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:37.173+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:37.209 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 51, 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "4f532d31-28c1-4236-a74e-6392960baf2b", 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2024-09-15T00:08:32.007191+0000", 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2024-09-15T00:09:36.926662+0000", 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2024-09-15T00:08:42.794810+0000", 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2024-09-15T00:08:33.890963+0000", 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5801984, 2024-09-15T00:09:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "noscrub", 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 3, 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 6, 2024-09-15T00:09:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "squid", 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:42.896076+0000", 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "14", 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2024-09-15T00:09:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2024-09-15T00:09:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2024-09-15T00:09:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 6.059999942779541, 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 6.059999942779541, 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.33000001311302185, 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 2, 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "rbd", 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:49.173060+0000", 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 8, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 8, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 8, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 8, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 8, 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "18", 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 18, 2024-09-15T00:09:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2024-09-15T00:09:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "pool_client.0", 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:58.879058+0000", 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:09:37.243 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:09:37.244 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:09:37.245 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:09:37.245 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:09:37.245 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:09:37.245 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:09:37.245 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:09:37.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "51", 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 20, 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 51, 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:09:37.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:09:37.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:09:37.248 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "56600dc2-b95d-47a7-b2b1-5ddc54c08341", 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:37.250 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:37.251 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2024-09-15T00:09:37.252 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.253 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/1440638768", 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/1440638768", 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6823/1440638768", 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6821/1440638768", 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:37.255 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "ea8f0155-8472-4e87-89fd-7716d05065d0", 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:37.256 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:37.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2024-09-15T00:09:37.258 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.259 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/3601510193", 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/3601510193", 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/3601510193", 2024-09-15T00:09:37.261 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/3601510193", 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "357d2d59-66f0-47b8-87c8-4acdfe62a07d", 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:37.262 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 26, 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:37.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/3222759256", 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/3222759256", 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/3222759256", 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/3222759256", 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "22208f56-d8aa-4a56-a42c-5a990962b860", 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:37.268 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6809", 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6811", 2024-09-15T00:09:37.269 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6813", 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6816", 2024-09-15T00:09:37.270 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6822", 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.271 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6823", 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6818", 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6821", 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6811/675428777", 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6816/675428777", 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6823/675428777", 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6821/675428777", 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "a974629e-9ef0-40a3-94bd-382e0bfa8bb6", 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:37.274 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6808", 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6810", 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6812", 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6814", 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.276 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6819", 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6820", 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.277 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6815", 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6817", 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.278 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6810/3736884138", 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6814/3736884138", 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6820/3736884138", 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6817/3736884138", 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:37.279 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "06b73c2d-466a-4c4e-89fa-31d6eeb1af1d", 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 44, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6800", 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6801", 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6802", 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6803", 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6806", 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6807", 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:09:37.283 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6804", 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6805", 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6801/2235671634", 2024-09-15T00:09:37.284 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6803/2235671634", 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6807/2235671634", 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6805/2235671634", 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:37.285 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.730107+0000", 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:37.286 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.365297+0000", 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.868500+0000", 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.788781+0000", 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:37.288 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.892143+0000", 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:09:37.289 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.990037+0000", 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.3", 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 0, 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 4 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.1d", 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 4, 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 2 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.1e", 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2024-09-15T00:09:37.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:37.292 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 3, 2024-09-15T00:09:37.292 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 4 2024-09-15T00:09:37.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.292 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:37.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.292 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:37.292 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2024-09-15T00:09:37.292 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2024-09-15T00:09:37.292 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2024-09-15T00:09:37.293 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:37.294 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:37.296 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:37.296 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:37.297 INFO:tasks.thrashosds.thrasher:j is {'epoch': 51, 'fsid': '4f532d31-28c1-4236-a74e-6392960baf2b', 'created': '2024-09-15T00:08:32.007191+0000', 'modified': '2024-09-15T00:09:36.926662+0000', 'last_up_change': '2024-09-15T00:08:42.794810+0000', 'last_in_change': '2024-09-15T00:08:33.890963+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:08:42.896076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:08:49.173060+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:08:58.879058+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 20, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '56600dc2-b95d-47a7-b2b1-5ddc54c08341', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 1440638768}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 1440638768}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 1440638768}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 1440638768}]}, 'public_addr': '172.21.15.27:6817/1440638768', 'cluster_addr': '172.21.15.27:6819/1440638768', 'heartbeat_back_addr': '172.21.15.27:6823/1440638768', 'heartbeat_front_addr': '172.21.15.27:6821/1440638768', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'ea8f0155-8472-4e87-89fd-7716d05065d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 3601510193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 3601510193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 3601510193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 3601510193}]}, 'public_addr': '172.21.15.27:6809/3601510193', 'cluster_addr': '172.21.15.27:6811/3601510193', 'heartbeat_back_addr': '172.21.15.27:6815/3601510193', 'heartbeat_front_addr': '172.21.15.27:6813/3601510193', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '357d2d59-66f0-47b8-87c8-4acdfe62a07d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 3222759256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 3222759256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 3222759256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 3222759256}]}, 'public_addr': '172.21.15.27:6801/3222759256', 'cluster_addr': '172.21.15.27:6803/3222759256', 'heartbeat_back_addr': '172.21.15.27:6807/3222759256', 'heartbeat_front_addr': '172.21.15.27:6805/3222759256', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '22208f56-d8aa-4a56-a42c-5a990962b860', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6809', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 675428777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6813', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6816', 'nonce': 675428777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 675428777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 675428777}]}, 'public_addr': '172.21.15.79:6811/675428777', 'cluster_addr': '172.21.15.79:6816/675428777', 'heartbeat_back_addr': '172.21.15.79:6823/675428777', 'heartbeat_front_addr': '172.21.15.79:6821/675428777', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a974629e-9ef0-40a3-94bd-382e0bfa8bb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6810', 'nonce': 3736884138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6814', 'nonce': 3736884138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6819', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6820', 'nonce': 3736884138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6815', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 3736884138}]}, 'public_addr': '172.21.15.79:6810/3736884138', 'cluster_addr': '172.21.15.79:6814/3736884138', 'heartbeat_back_addr': '172.21.15.79:6820/3736884138', 'heartbeat_front_addr': '172.21.15.79:6817/3736884138', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '06b73c2d-466a-4c4e-89fa-31d6eeb1af1d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 2235671634}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2235671634}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2235671634}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2235671634}]}, 'public_addr': '172.21.15.79:6801/2235671634', 'cluster_addr': '172.21.15.79:6803/2235671634', 'heartbeat_back_addr': '172.21.15.79:6807/2235671634', 'heartbeat_front_addr': '172.21.15.79:6805/2235671634', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.730107+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.365297+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.868500+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.788781+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.892143+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.990037+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.3', 'mappings': [{'from': 0, 'to': 4}]}, {'pgid': '3.1d', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.1e', 'mappings': [{'from': 3, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:09:37.297 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.3 2024-09-15T00:09:37.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.3 2024-09-15T00:09:37.298 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:37.293+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:37.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:37.333 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:37.333 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:37.352 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:37.352 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:37.369 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:37.369 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:37.386 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:37.386 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:37.396 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:37.394+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:37.401 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:37.402 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:37.404 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:37.404 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:37.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:37.421 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:37.421 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:37.497 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:37.495+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:37.520 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:37.520 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:37.526 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-15T00:09:37.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:37.612 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:37.611+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:37.685 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:37.685 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:37.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:37.685 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:37.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.685 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3427 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.685 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:37.685 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394812242, 2024-09-15T00:09:37.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027724631, 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3427 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287582+0000", 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2599999999999998e-07 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287584+0000", 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8759999999999999e-06 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287589+0000", 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287594+0000", 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.357e-06 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315217+0000", 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027622785 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315257+0000", 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0228000000000002e-05 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315306+0000", 2024-09-15T00:09:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9228999999999999e-05 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3428 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394794236999999, 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027928562000000001, 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3428 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287600+0000", 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287602+0000", 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287608+0000", 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287613+0000", 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8740000000000003e-06 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315346+0000", 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027733099000000001 2024-09-15T00:09:37.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315431+0000", 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4350000000000004e-05 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315528+0000", 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7374000000000007e-05 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3429 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394774927, 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028028630999999998, 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3429 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287621+0000", 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287627+0000", 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8440000000000003e-06 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287631+0000", 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5009999999999998e-06 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315556+0000", 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027925241999999999 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315595+0000", 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8512e-05 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315647+0000", 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2318000000000003e-05 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3430 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.699 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394756853000001, 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028122114, 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3430 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:37.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287644+0000", 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4219999999999997e-06 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287650+0000", 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0920000000000003e-06 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287654+0000", 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4279999999999998e-06 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315671+0000", 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016592 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315709+0000", 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8231e-05 2024-09-15T00:09:37.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315759+0000", 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9926999999999998e-05 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3431 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394733313, 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02831842, 2024-09-15T00:09:37.704 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3431 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287661+0000", 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287663+0000", 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9570000000000001e-06 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287668+0000", 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287673+0000", 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:09:37.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315799+0000", 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028125741999999999 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315891+0000", 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2701000000000006e-05 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315979+0000", 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7287000000000006e-05 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3432 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394714928999999, 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028430624000000002, 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3432 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287681+0000", 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8530000000000001e-06 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287686+0000", 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4659999999999998e-06 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287691+0000", 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2230000000000001e-06 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316006+0000", 2024-09-15T00:09:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028315344999999999 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316054+0000", 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8004999999999999e-05 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316109+0000", 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5374999999999998e-05 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3433 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394697173000001, 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028528593000000001, 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3433 2024-09-15T00:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287697+0000", 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287699+0000", 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8509999999999999e-06 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287704+0000", 2024-09-15T00:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287710+0000", 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316124+0000", 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028414466999999999 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316171+0000", 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6783999999999998e-05 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316225+0000", 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3891999999999998e-05 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3434 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394677787999999, 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028632801999999999, 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3434 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-07 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287718+0000", 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287724+0000", 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8390000000000003e-06 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287729+0000", 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8389999999999997e-06 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316250+0000", 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028521135 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316295+0000", 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4540999999999999e-05 2024-09-15T00:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316349+0000", 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4083999999999999e-05 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3435 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394647112000001, 2024-09-15T00:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028723243999999998, 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3435 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287747+0000", 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4900000000000002e-07 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287748+0000", 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4190000000000001e-06 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287752+0000", 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7900000000000001e-06 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287756+0000", 2024-09-15T00:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1609999999999996e-06 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316375+0000", 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028618932 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316417+0000", 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316470+0000", 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3436 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394632064, 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028827854, 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3436 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287766+0000", 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287771+0000", 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5140000000000001e-06 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287776+0000", 2024-09-15T00:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316494+0000", 2024-09-15T00:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028718144000000001 2024-09-15T00:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316538+0000", 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4129999999999999e-05 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316589+0000", 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1548999999999997e-05 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3437 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394614172000001, 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028972188999999999, 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3437 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287780+0000", 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287782+0000", 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8419999999999999e-06 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287787+0000", 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3059999999999997e-06 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287791+0000", 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9920000000000001e-06 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316615+0000", 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028824487999999999 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316668+0000", 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2883999999999997e-05 2024-09-15T00:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316752+0000", 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3308999999999998e-05 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3438 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394597130000001, 2024-09-15T00:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029124331999999999, 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3438 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287797+0000", 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287799+0000", 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287804+0000", 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7069999999999997e-06 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287808+0000", 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0720000000000001e-06 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316799+0000", 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028990241999999999 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316845+0000", 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5837000000000002e-05 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316921+0000", 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6255999999999996e-05 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3439 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394579590999999, 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029290665, 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3439 2024-09-15T00:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287816+0000", 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6980000000000001e-06 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287824+0000", 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2500000000000006e-06 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287829+0000", 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316967+0000", 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029138319999999999 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317047+0000", 2024-09-15T00:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0081999999999997e-05 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317105+0000", 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7497999999999997e-05 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3440 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394558532000001, 2024-09-15T00:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029395451999999999, 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3440 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287837+0000", 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287843+0000", 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6799999999999998e-06 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287848+0000", 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317131+0000", 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029283337999999999 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317176+0000", 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5176999999999998e-05 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317231+0000", 2024-09-15T00:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4120999999999999e-05 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3441 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394540467999999, 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029496321999999998, 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3441 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287855+0000", 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7099999999999999e-06 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287861+0000", 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287866+0000", 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317252+0000", 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029386051999999999 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317297+0000", 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5164999999999999e-05 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317349+0000", 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2534000000000002e-05 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3442 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394521202, 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029594697999999999, 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3442 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287873+0000", 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-15T00:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287879+0000", 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287885+0000", 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.755e-06 2024-09-15T00:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287890+0000", 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317375+0000", 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029485457 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317416+0000", 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1217999999999997e-05 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317467+0000", 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0819999999999998e-05 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3443 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394497863000002, 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029684676, 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3443 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287898+0000", 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287904+0000", 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8860000000000001e-06 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287908+0000", 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5650000000000003e-06 2024-09-15T00:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317488+0000", 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029579437 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317530+0000", 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1900999999999997e-05 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317580+0000", 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0603000000000003e-05 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3444 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394472645, 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029772823, 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3444 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287931+0000", 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0008e-05 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287938+0000", 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287943+0000", 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317603+0000", 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029659913999999999 2024-09-15T00:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317646+0000", 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2367999999999998e-05 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317694+0000", 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8201999999999997e-05 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3445 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394443307, 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029863565000000002, 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3445 2024-09-15T00:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287951+0000", 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287953+0000", 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.765e-06 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287958+0000", 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999996e-06 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287963+0000", 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.634e-06 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317714+0000", 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029751353000000001 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317755+0000", 2024-09-15T00:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0342000000000001e-05 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317814+0000", 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9188999999999999e-05 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3446 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.394425008999999, 2024-09-15T00:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029967884, 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3446 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287971+0000", 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287977+0000", 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4330000000000003e-06 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287981+0000", 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3989999999999997e-06 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317837+0000", 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029855942 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317878+0000", 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467000000000001e-05 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317936+0000", 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8100999999999998e-05 2024-09-15T00:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:37.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:37.782 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:37.780+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:37.782 INFO:teuthology.orchestra.run.smithi092.stdout:1906 read 0x4611600 thru 0x461afc1 (0x99c2 bytes) 2024-09-15T00:09:37.788 INFO:teuthology.orchestra.run.smithi092.stdout:1907 trunc from 0xc725600 to 0x7b12000 2024-09-15T00:09:37.825 INFO:teuthology.orchestra.run.smithi092.stdout:1908 read 0x6133600 thru 0x6136d7b (0x377c bytes) 2024-09-15T00:09:37.827 INFO:teuthology.orchestra.run.smithi092.stdout:1909 punch from 0x7730200 to 0x773b600, (0xb400 bytes) 2024-09-15T00:09:37.830 INFO:teuthology.orchestra.run.smithi092.stdout:1911 write 0x58d6400 thru 0x58e211f (0xbd20 bytes) 2024-09-15T00:09:37.858 INFO:teuthology.orchestra.run.smithi092.stdout:1912 write 0x75e4600 thru 0x75f3828 (0xf229 bytes) 2024-09-15T00:09:37.882 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:37.880+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:37.903 INFO:teuthology.orchestra.run.smithi092.stdout:1914 write 0xce7dc00 thru 0xce83a61 (0x5e62 bytes) 2024-09-15T00:09:37.905 INFO:teuthology.orchestra.run.smithi092.stdout:1916 read 0xaa2fc00 thru 0xaa3e73e (0xeb3f bytes) 2024-09-15T00:09:37.905 INFO:teuthology.orchestra.run.smithi092.stdout:1918 punch from 0x143e600 to 0x1449a00, (0xb400 bytes) 2024-09-15T00:09:37.911 INFO:teuthology.orchestra.run.smithi092.stdout:1921 read 0x9fc5e00 thru 0x9fd1a79 (0xbc7a bytes) 2024-09-15T00:09:37.937 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:37.937 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:37.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:09:37.942 INFO:teuthology.orchestra.run.smithi027.stderr:clear 3.3 pg_upmap_items mapping 2024-09-15T00:09:37.946 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:37.966 INFO:teuthology.orchestra.run.smithi092.stdout:1926 write 0xd737e00 thru 0xd73c379 (0x457a bytes) 2024-09-15T00:09:37.966 INFO:teuthology.orchestra.run.smithi092.stdout:1927 trunc from 0xd73c400 to 0xd6ffe00 2024-09-15T00:09:37.982 INFO:teuthology.orchestra.run.smithi092.stdout:1929 read 0x1816400 thru 0x181885b (0x245c bytes) 2024-09-15T00:09:37.984 INFO:teuthology.orchestra.run.smithi092.stdout:1935 write 0x9b9de00 thru 0x9bac3c7 (0xe5c8 bytes) 2024-09-15T00:09:37.984 INFO:teuthology.orchestra.run.smithi092.stdout:1936 read 0xb926600 thru 0xb926b8c (0x58d bytes) 2024-09-15T00:09:37.984 INFO:teuthology.orchestra.run.smithi092.stdout:1937 write 0x6076400 thru 0x6085f80 (0xfb81 bytes) 2024-09-15T00:09:37.985 INFO:teuthology.orchestra.run.smithi092.stdout:1938 write 0x41f5200 thru 0x4204de0 (0xfbe1 bytes) 2024-09-15T00:09:37.985 INFO:teuthology.orchestra.run.smithi092.stdout:1939 punch from 0x63bda00 to 0x63bea00, (0x1000 bytes) 2024-09-15T00:09:37.990 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:37.989+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:38.009 INFO:teuthology.orchestra.run.smithi092.stdout:1940 trunc from 0xd6ffe00 to 0xb7c0000 2024-09-15T00:09:38.078 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:38.078 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:38.083 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:38.081+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:38.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:38.132 INFO:teuthology.orchestra.run.smithi092.stdout:1942 trunc from 0xb7c0000 to 0x1929c00 2024-09-15T00:09:38.185 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:38.183+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:38.206 INFO:teuthology.orchestra.run.smithi092.stdout:1943 read 0x1627000 thru 0x1631956 (0xa957 bytes) 2024-09-15T00:09:38.209 INFO:teuthology.orchestra.run.smithi092.stdout:1944 read 0x15e3400 thru 0x15ed86d (0xa46e bytes) 2024-09-15T00:09:38.210 INFO:teuthology.orchestra.run.smithi092.stdout:1946 read 0x11e8800 thru 0x11ebe7b (0x367c bytes) 2024-09-15T00:09:38.218 INFO:teuthology.orchestra.run.smithi092.stdout:1947 trunc from 0x1929c00 to 0x7b4a000 2024-09-15T00:09:38.227 INFO:teuthology.orchestra.run.smithi092.stdout:1948 read 0x2e96c00 thru 0x2e98639 (0x1a3a bytes) 2024-09-15T00:09:38.228 INFO:teuthology.orchestra.run.smithi092.stdout:1949 read 0x534c00 thru 0x540b87 (0xbf88 bytes) 2024-09-15T00:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1142 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.352648429999999, 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045190363999999997, 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1142 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882756+0000", 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882773+0000", 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6872999999999999e-05 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882780+0000", 2024-09-15T00:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4240000000000004e-06 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882787+0000", 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8510000000000001e-06 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927799+0000", 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045011974000000003 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927843+0000", 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4270999999999997e-05 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927946+0000", 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102581 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.352599737999999, 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.352561108, 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.352535171, 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.352519102999999, 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.352378007999999, 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.351606253, 2024-09-15T00:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.351521221999999, 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.351486206000001, 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.351460603, 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.351286337000001, 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.351055198000001, 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1157 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.35027376, 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045184281, 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1157 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885132+0000", 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885139+0000", 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0350000000000004e-06 2024-09-15T00:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885153+0000", 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3515e-05 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930177+0000", 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045024495999999997 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930249+0000", 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2072999999999999e-05 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930314+0000", 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4826999999999994e-05 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.350243253999999, 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.350163513999998, 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.349958488999999, 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.349746956000001, 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.347190949000002, 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:09:38.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.345207843000001, 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.329 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.345188272000001, 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:09:38.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:38.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:38.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:09:38.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:09:38.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:38.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:38.334 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:38.335 INFO:teuthology.orchestra.run.smithi092.stdout:1950 write 0xc374000 thru 0xc3814a8 (0xd4a9 bytes) 2024-09-15T00:09:38.336 INFO:teuthology.orchestra.run.smithi092.stdout:1951 read 0x237a000 thru 0x2389294 (0xf295 bytes) 2024-09-15T00:09:38.336 INFO:teuthology.orchestra.run.smithi092.stdout:1952 clone 19 order 24 su 4194304 sc 4 2024-09-15T00:09:38.336 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:38.334+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:38.336 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:38.436 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:38.434+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:38.444 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:38.444 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:38.444 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:38.444 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:38.452 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:38.537 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:38.535+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:38.601 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:38.609 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:38.639 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:38.637+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:38.739 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:38.737+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:38.757 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8832 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.602247952999999, 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056083069999999999, 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:38.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8832 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153006+0000", 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2099999999999998e-07 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153008+0000", 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.187e-06 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153012+0000", 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8909999999999997e-06 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153015+0000", 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8559999999999998e-06 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202369+0000", 2024-09-15T00:09:38.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.049354393000000003 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202431+0000", 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2235000000000003e-05 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.202749+0000", 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00031714299999999998 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204050+0000", 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00130164 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209033+0000", 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049826590000000004 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209063+0000", 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0176999999999999e-05 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209089+0000", 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5568e-05 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8833 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.602233473999998, 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056156170999999998, 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8833 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153020+0000", 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1299999999999999e-07 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153022+0000", 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5829999999999999e-06 2024-09-15T00:09:38.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153025+0000", 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.022e-06 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153029+0000", 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8500000000000004e-06 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203131+0000", 2024-09-15T00:09:38.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.05010212 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203216+0000", 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.543e-05 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.203493+0000", 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027699000000000002 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206606+0000", 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003112315 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209120+0000", 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002514863 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209149+0000", 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8685000000000001e-05 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209176+0000", 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7100000000000001e-05 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8834 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.602221931999999, 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056218377999999999, 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8834 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153032+0000", 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.16e-07 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153034+0000", 2024-09-15T00:09:38.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.844e-06 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153039+0000", 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8369999999999996e-06 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153041+0000", 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5620000000000002e-06 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204094+0000", 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051053378000000003 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204144+0000", 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9907999999999997e-05 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204342+0000", 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000197596 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206613+0000", 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022710579999999998 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209204+0000", 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0025907780000000002 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209230+0000", 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6041999999999999e-05 2024-09-15T00:09:38.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209250+0000", 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0159000000000001e-05 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8835 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.602209521999999, 2024-09-15T00:09:38.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060077884999999998, 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8835 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153044+0000", 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.16e-07 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153046+0000", 2024-09-15T00:09:38.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153049+0000", 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153051+0000", 2024-09-15T00:09:38.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.52e-06 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204715+0000", 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.051664032999999998 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.204804+0000", 2024-09-15T00:09:38.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8383999999999999e-05 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205194+0000", 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039034399999999999 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206890+0000", 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016954680000000001 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213068+0000", 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.006178411 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213097+0000", 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9433000000000001e-05 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213122+0000", 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4553999999999999e-05 2024-09-15T00:09:38.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8836 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.602199136999999, 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060148357, 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:38.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8836 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153054+0000", 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153055+0000", 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0100000000000001e-07 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153056+0000", 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6220000000000001e-06 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153060+0000", 2024-09-15T00:09:38.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5499999999999999e-06 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153062+0000", 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6510000000000001e-06 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205724+0000", 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.052661396999999999 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.205802+0000", 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8668e-05 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206109+0000", 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030663400000000002 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.208482+0000", 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023725019999999999 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213151+0000", 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046689660000000001 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213181+0000", 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0324999999999998e-05 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213203+0000", 2024-09-15T00:09:38.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1841e-05 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8837 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.602188263999999, 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060386983999999998, 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:38.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8837 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:38.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153065+0000", 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1500000000000001e-07 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153067+0000", 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.764e-06 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153070+0000", 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.869e-06 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153073+0000", 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.548e-06 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206907+0000", 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053834290999999999 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.206953+0000", 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5648000000000001e-05 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207282+0000", 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032939999999999998 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209097+0000", 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018149170000000001 2024-09-15T00:09:38.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213388+0000", 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042910989999999996 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213430+0000", 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2465000000000001e-05 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213452+0000", 2024-09-15T00:09:38.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1767999999999999e-05 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4622.0:8838 3.12 3:4ae48df8:::rbd_object_map.112981949474:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.602178022, 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060498379999999997, 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:09:38.795 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4622", 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8838 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153076+0000", 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.04e-07 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153077+0000", 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.389e-06 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153080+0000", 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8499999999999998e-06 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.153083+0000", 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.565e-06 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207872+0000", 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.054789110000000002 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.207926+0000", 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4515000000000002e-05 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 3", 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.208143+0000", 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000216784 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.209511+0000", 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013684350000000001 2024-09-15T00:09:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213494+0000", 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039824889999999996 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213535+0000", 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0865999999999998e-05 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:21.213574+0000", 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9172999999999999e-05 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3358 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7684096~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7090836389999993, 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055739138000000001, 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3358 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046170+0000", 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046177+0000", 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7000000000000002e-06 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046184+0000", 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.029e-06 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046189+0000", 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9459999999999997e-06 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101686+0000", 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055497367999999998 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101778+0000", 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2077000000000006e-05 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101909+0000", 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013064700000000001 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.805 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3359 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7733248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7090558950000005, 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055953638999999999, 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3359 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046198+0000", 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:09:38.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046200+0000", 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046207+0000", 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8050000000000001e-06 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046213+0000", 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3489999999999999e-06 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.101957+0000", 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055744316000000002 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102048+0000", 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1464000000000004e-05 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102151+0000", 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102874 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3360 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7782400~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7090334840000008, 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056128863000000001, 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3360 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046220+0000", 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046223+0000", 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2230000000000001e-06 2024-09-15T00:09:38.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046229+0000", 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6839999999999999e-06 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046234+0000", 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6800000000000001e-06 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102187+0000", 2024-09-15T00:09:38.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.05595282 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102262+0000", 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5371999999999996e-05 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102349+0000", 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6706999999999995e-05 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3361 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7831552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.70901265, 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056322561, 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3361 2024-09-15T00:09:38.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046241+0000", 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046244+0000", 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.43e-06 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046250+0000", 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5420000000000002e-06 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046255+0000", 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.707e-06 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102382+0000", 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056126796999999999 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102469+0000", 2024-09-15T00:09:38.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7023999999999999e-05 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102563+0000", 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4678000000000004e-05 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3362 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7880704~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046262+0000", 2024-09-15T00:09:38.819 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7089916079999998, 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056530960999999998, 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3362 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046262+0000", 2024-09-15T00:09:38.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046262+0000", 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046492+0000", 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023003100000000001 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046496+0000", 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9210000000000002e-06 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046506+0000", 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0001e-05 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046513+0000", 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9229999999999996e-06 2024-09-15T00:09:38.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102610+0000", 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056097039000000001 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102698+0000", 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7860000000000002e-05 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102793+0000", 2024-09-15T00:09:38.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5186000000000006e-05 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3363 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7929856~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046548+0000", 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7087053730000008, 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056535131000000002, 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3363 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046548+0000", 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046548+0000", 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046549+0000", 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6000000000000003e-07 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046552+0000", 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3900000000000002e-06 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046564+0000", 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1474999999999999e-05 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046572+0000", 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4009999999999997e-06 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102824+0000", 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056251511999999997 2024-09-15T00:09:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.102922+0000", 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8499999999999995e-05 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103083+0000", 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016119300000000001 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3364 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 7979008~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.046606+0000", 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7086471260000007, 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056797145, 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3364 2024-09-15T00:09:38.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046606+0000", 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.046606+0000", 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047399+0000", 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00079283299999999995 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047403+0000", 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1670000000000001e-06 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047413+0000", 2024-09-15T00:09:38.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4220000000000006e-06 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047419+0000", 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4450000000000001e-06 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103137+0000", 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055717322999999999 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103252+0000", 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115701 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103404+0000", 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000151254 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3365 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8028160~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.833 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.047431+0000", 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7078225200000006, 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056539155000000001, 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3365 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047431+0000", 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047431+0000", 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047432+0000", 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047434+0000", 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047441+0000", 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.849e-06 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047447+0000", 2024-09-15T00:09:38.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.609e-06 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103694+0000", 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056247765999999998 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103808+0000", 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011345699999999999 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.103970+0000", 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000162421 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3366 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8077312~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.047453+0000", 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7078000749999998, 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056741575000000002, 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3366 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047453+0000", 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047453+0000", 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047454+0000", 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047456+0000", 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.125e-06 2024-09-15T00:09:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047462+0000", 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5999999999999997e-06 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047466+0000", 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5990000000000004e-06 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104001+0000", 2024-09-15T00:09:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056535215 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104093+0000", 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1496999999999995e-05 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104195+0000", 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010213900000000001 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3367 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8126464~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.047473+0000", 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7077808000000001, 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056984806999999998, 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3367 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047473+0000", 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.047473+0000", 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049398+0000", 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001925689 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049403+0000", 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7700000000000001e-06 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049415+0000", 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1657999999999999e-05 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049422+0000", 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6350000000000006e-06 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104236+0000", 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.054813957000000003 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104352+0000", 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011547100000000001 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.104458+0000", 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105627 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3370 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8273920~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7057347659999991, 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055967186000000002, 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3370 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049519+0000", 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9999999999999997e-07 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049522+0000", 2024-09-15T00:09:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5900000000000002e-06 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049531+0000", 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0270000000000001e-06 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049537+0000", 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8610000000000003e-06 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105279+0000", 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055742236000000001 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105364+0000", 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4993000000000004e-05 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105486+0000", 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:09:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3371 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8323072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.049545+0000", 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7057082290000007, 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056146330000000001, 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3371 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049545+0000", 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049545+0000", 2024-09-15T00:09:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049546+0000", 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7300000000000001e-07 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049548+0000", 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5770000000000001e-06 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049740+0000", 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019119300000000001 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049749+0000", 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3850000000000008e-06 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105519+0000", 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055770239999999999 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105601+0000", 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1460999999999998e-05 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105692+0000", 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1001000000000005e-05 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3372 3.1b 3:dda78d0d:::rbd_data.1129915ddc5d.0000000000000002:f [read 8372224~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.7054888639999994, 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.056196995, 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3372 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049765+0000", 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9100000000000004e-07 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049769+0000", 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8399999999999997e-06 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049777+0000", 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2800000000000003e-06 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.049783+0000", 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4790000000000004e-06 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105753+0000", 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.055970585000000003 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105831+0000", 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7171999999999999e-05 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.105962+0000", 2024-09-15T00:09:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013104800000000001 2024-09-15T00:09:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:38.861 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:38.863 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:38.863 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:38.860+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:38.944 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:09:39.001 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:38.999+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:39.018 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:39.018 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:39.018 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:39.018 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:39.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:39.028 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone18 from 0xc381600 (overlap 0x1929c00) to 0x0 2024-09-15T00:09:39.068 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:39.066+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:39.166 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:39.164+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:39.217 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:39.218 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:39.218 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:39.218 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:39.218 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:39.233 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:39.267 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:39.265+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:39.368 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:39.367+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3731008659999997, 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.407 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3730806110000007, 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.408 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.412 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3730608649999994, 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:39.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:09:39.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3730409150000007, 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:09:39.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3730134150000008, 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.422 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:09:39.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3729965740000001, 2024-09-15T00:09:39.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:09:39.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:09:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3729805759999998, 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.431 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:39.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.435 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3729575880000002, 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:09:39.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3729317039999991, 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:09:39.440 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:09:39.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.444 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3729152770000006, 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.445 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:39.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3729010059999993, 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.450 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:39.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:09:39.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3728916889999994, 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:09:39.454 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:09:39.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3712226770000004, 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:09:39.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:39.463 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3711876140000001, 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:09:39.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:09:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3711625870000006, 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.468 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:09:39.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3711382790000002, 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:09:39.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:09:39.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2674 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3768320~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3689919330000002, 2024-09-15T00:09:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059850591000000002, 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2674 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031671+0000", 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031682+0000", 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1248e-05 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031693+0000", 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1395e-05 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031705+0000", 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2106000000000001e-05 2024-09-15T00:09:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091352+0000", 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059646421999999998 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091441+0000", 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9174999999999999e-05 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091521+0000", 2024-09-15T00:09:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9636999999999997e-05 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3689460279999999, 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:09:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3689218959999998, 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3688925239999996, 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:09:39.491 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:09:39.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:39.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:39.496 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:39.496 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:39.496 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:39.498 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:39.498 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:39.495+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:39.598 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:39.596+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:39.653 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:39.653 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:39.653 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:39.653 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:39.656 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-15T00:09:39.662 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:39.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:39.668 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:39.690 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:39.690 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:39.699 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:39.698+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:39.783 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:39.783 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:39.783 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:39.783 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:39.783 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:39.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:39.799 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:39.798+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:39.855 INFO:teuthology.orchestra.run.smithi092.stdout:1955 read 0x5364400 thru 0x536d02d (0x8c2e bytes) 2024-09-15T00:09:39.856 INFO:teuthology.orchestra.run.smithi092.stdout:1958 punch from 0x9c2fa00 to 0x9c38a00, (0x9000 bytes) 2024-09-15T00:09:39.864 INFO:teuthology.orchestra.run.smithi092.stdout:1960 write 0x776a400 thru 0x776f820 (0x5421 bytes) 2024-09-15T00:09:39.867 INFO:teuthology.orchestra.run.smithi092.stdout:1961 trunc from 0xc381600 to 0xed3a200 2024-09-15T00:09:39.881 INFO:teuthology.orchestra.run.smithi092.stdout:1962 read 0x845e200 thru 0x8468f8d (0xad8e bytes) 2024-09-15T00:09:39.895 INFO:teuthology.orchestra.run.smithi092.stdout:1964 trunc from 0xed3a200 to 0x741c600 2024-09-15T00:09:39.900 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:39.898+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:39.912 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:39.913 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:39.914 INFO:teuthology.orchestra.run.smithi092.stdout:1968 read 0x279c400 thru 0x27a6bf5 (0xa7f6 bytes) 2024-09-15T00:09:39.924 INFO:teuthology.orchestra.run.smithi092.stdout:1969 trunc from 0x741c600 to 0xb4fe400 2024-09-15T00:09:39.932 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:39.932 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:39.936 INFO:teuthology.orchestra.run.smithi092.stdout:1970 trunc from 0xb4fe400 to 0xda84800 2024-09-15T00:09:39.952 INFO:teuthology.orchestra.run.smithi092.stdout:1977 trunc from 0xda84800 to 0xe21c600 2024-09-15T00:09:39.954 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:39.954 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:39.955 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:39.955 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:39.965 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:39.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:39.974 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:39.986 INFO:teuthology.orchestra.run.smithi092.stdout:1980 trunc from 0xe21c600 to 0x1bd0c00 2024-09-15T00:09:40.008 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:40.007+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:40.070 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:40.071 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:40.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:40.101 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:40.099+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:40.151 INFO:teuthology.orchestra.run.smithi092.stdout:1981 trunc from 0x1bd0c00 to 0x6d8b600 2024-09-15T00:09:40.159 INFO:teuthology.orchestra.run.smithi092.stdout:1983 read 0x541200 thru 0x550cd2 (0xfad3 bytes) 2024-09-15T00:09:40.167 INFO:teuthology.orchestra.run.smithi092.stdout:1984 read 0x4451e00 thru 0x445a292 (0x8493 bytes) 2024-09-15T00:09:40.191 INFO:teuthology.orchestra.run.smithi092.stdout:1987 write 0xcc7cc00 thru 0xcc89fb1 (0xd3b2 bytes) 2024-09-15T00:09:40.191 INFO:teuthology.orchestra.run.smithi092.stdout:1988 write 0xa73ba00 thru 0xa73fdcc (0x43cd bytes) 2024-09-15T00:09:40.191 INFO:teuthology.orchestra.run.smithi092.stdout:1991 write 0xc253a00 thru 0xc262937 (0xef38 bytes) 2024-09-15T00:09:40.191 INFO:teuthology.orchestra.run.smithi092.stdout:1993 trunc from 0xcc8a000 to 0xcd0c200 2024-09-15T00:09:40.202 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:40.201+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:40.223 INFO:teuthology.orchestra.run.smithi092.stdout:1994 write 0xe294400 thru 0xe297e98 (0x3a99 bytes) 2024-09-15T00:09:40.227 INFO:teuthology.orchestra.run.smithi092.stdout:1995 trunc from 0xe298000 to 0xc017600 2024-09-15T00:09:40.228 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:40.228 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:40.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:40.244 INFO:teuthology.orchestra.run.smithi092.stdout:1996 write 0x554a400 thru 0x55583a2 (0xdfa3 bytes) 2024-09-15T00:09:40.244 INFO:teuthology.orchestra.run.smithi092.stdout:1999 punch from 0x1478000 to 0x1486800, (0xe800 bytes) 2024-09-15T00:09:40.255 INFO:teuthology.orchestra.run.smithi092.stdout:2001 trunc from 0xc017600 to 0x3c59600 2024-09-15T00:09:40.269 INFO:teuthology.orchestra.run.smithi092.stdout:2002 read 0x57e600 thru 0x58b5d1 (0xcfd2 bytes) 2024-09-15T00:09:40.292 INFO:teuthology.orchestra.run.smithi092.stdout:2004 write 0xece4800 thru 0xecea99b (0x619c bytes) 2024-09-15T00:09:40.293 INFO:teuthology.orchestra.run.smithi092.stdout:2005 punch from 0xcaf8400 to 0xcaf9800, (0x1400 bytes) 2024-09-15T00:09:40.295 INFO:teuthology.orchestra.run.smithi092.stdout:2008 read 0xb709200 thru 0xb70a088 (0xe89 bytes) 2024-09-15T00:09:40.295 INFO:teuthology.orchestra.run.smithi092.stdout:2009 punch from 0x97d0000 to 0x97d0a00, (0xa00 bytes) 2024-09-15T00:09:40.295 INFO:teuthology.orchestra.run.smithi092.stdout:2010 read 0xbd7c000 thru 0xbd8a66a (0xe66b bytes) 2024-09-15T00:09:40.295 INFO:teuthology.orchestra.run.smithi092.stdout:2011 write 0x70d9200 thru 0x70deddc (0x5bdd bytes) 2024-09-15T00:09:40.295 INFO:teuthology.orchestra.run.smithi092.stdout:2012 read 0xb20c000 thru 0xb218dfe (0xcdff bytes) 2024-09-15T00:09:40.295 INFO:teuthology.orchestra.run.smithi092.stdout:2013 read 0x3ec0400 thru 0x3ec082f (0x430 bytes) 2024-09-15T00:09:40.296 INFO:teuthology.orchestra.run.smithi092.stdout:2014 write 0x7d2e600 thru 0x7d2eaab (0x4ac bytes) 2024-09-15T00:09:40.296 INFO:teuthology.orchestra.run.smithi092.stdout:2015 read 0x243a000 thru 0x2446d8f (0xcd90 bytes) 2024-09-15T00:09:40.296 INFO:teuthology.orchestra.run.smithi092.stdout:2016 punch from 0x596aa00 to 0x596c800, (0x1e00 bytes) 2024-09-15T00:09:40.296 INFO:teuthology.orchestra.run.smithi092.stdout:2018 trunc from 0xeceaa00 to 0xe975a00 2024-09-15T00:09:40.311 INFO:teuthology.orchestra.run.smithi092.stdout:2023 read 0x6c6ee00 thru 0x6c6f74f (0x950 bytes) 2024-09-15T00:09:40.320 INFO:teuthology.orchestra.run.smithi092.stdout:2024 write 0xad7ae00 thru 0xad7c2fb (0x14fc bytes) 2024-09-15T00:09:40.320 INFO:teuthology.orchestra.run.smithi092.stdout:2025 write 0x58e9800 thru 0x58f53a2 (0xbba3 bytes) 2024-09-15T00:09:40.321 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:40.320+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:40.329 INFO:teuthology.orchestra.run.smithi092.stdout:2027 trunc from 0xe975a00 to 0x6ce1000 2024-09-15T00:09:40.347 INFO:teuthology.orchestra.run.smithi092.stdout:2028 write 0x6bc4800 thru 0x6bd34eb (0xecec bytes) 2024-09-15T00:09:40.354 INFO:teuthology.orchestra.run.smithi092.stdout:2029 read 0x2068e00 thru 0x206fcf3 (0x6ef4 bytes) 2024-09-15T00:09:40.355 INFO:teuthology.orchestra.run.smithi092.stdout:2033 read 0x2c47400 thru 0x2c51e8f (0xaa90 bytes) 2024-09-15T00:09:40.359 INFO:teuthology.orchestra.run.smithi092.stdout:2034 trunc from 0x6ce1000 to 0x3b5c200 2024-09-15T00:09:40.383 INFO:teuthology.orchestra.run.smithi092.stdout:2035 punch from 0x2ba6200 to 0x2ba6e00, (0xc00 bytes) 2024-09-15T00:09:40.389 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:40.389 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:40.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:40.401 INFO:teuthology.orchestra.run.smithi092.stdout:2040 write 0x9ea0000 thru 0x9eafc5c (0xfc5d bytes) 2024-09-15T00:09:40.402 INFO:teuthology.orchestra.run.smithi092.stdout:2041 read 0x8eca00 thru 0x8f3507 (0x6b08 bytes) 2024-09-15T00:09:40.403 INFO:teuthology.orchestra.run.smithi092.stdout:2044 write 0x2c53a00 thru 0x2c5fe7d (0xc47e bytes) 2024-09-15T00:09:40.403 INFO:teuthology.orchestra.run.smithi092.stdout:2045 punch from 0x7af8e00 to 0x7b04200, (0xb400 bytes) 2024-09-15T00:09:40.404 INFO:teuthology.orchestra.run.smithi092.stdout:2046 punch from 0x604da00 to 0x6058400, (0xaa00 bytes) 2024-09-15T00:09:40.404 INFO:teuthology.orchestra.run.smithi092.stdout:2048 write 0x575c000 thru 0x57606ad (0x46ae bytes) 2024-09-15T00:09:40.417 INFO:teuthology.orchestra.run.smithi092.stdout:2049 write 0xb9a6200 thru 0xb9b2f89 (0xcd8a bytes) 2024-09-15T00:09:40.428 INFO:teuthology.orchestra.run.smithi092.stdout:2050 trunc from 0xb9b3000 to 0x2cfb600 2024-09-15T00:09:40.440 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:40.438+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:40.461 INFO:teuthology.orchestra.run.smithi092.stdout:2054 write 0x9846400 thru 0x9847739 (0x133a bytes) 2024-09-15T00:09:40.461 INFO:teuthology.orchestra.run.smithi092.stdout:2055 read 0x4641600 thru 0x464dc37 (0xc638 bytes) 2024-09-15T00:09:40.462 INFO:teuthology.orchestra.run.smithi092.stdout:2060 write 0x6778200 thru 0x677f429 (0x722a bytes) 2024-09-15T00:09:40.462 INFO:teuthology.orchestra.run.smithi092.stdout:2062 punch from 0x86c1a00 to 0x86d0200, (0xe800 bytes) 2024-09-15T00:09:40.463 INFO:teuthology.orchestra.run.smithi092.stdout:2063 read 0x8272600 thru 0x827c1c1 (0x9bc2 bytes) 2024-09-15T00:09:40.463 INFO:teuthology.orchestra.run.smithi092.stdout:2066 read 0x1692000 thru 0x1696336 (0x4337 bytes) 2024-09-15T00:09:40.464 INFO:teuthology.orchestra.run.smithi092.stdout:2067 read 0x83fda00 thru 0x84009b2 (0x2fb3 bytes) 2024-09-15T00:09:40.468 INFO:teuthology.orchestra.run.smithi092.stdout:2068 trunc from 0x9847800 to 0xd34e400 2024-09-15T00:09:40.477 INFO:teuthology.orchestra.run.smithi092.stdout:2069 write 0x956a800 thru 0x9572a6b (0x826c bytes) 2024-09-15T00:09:40.477 INFO:teuthology.orchestra.run.smithi092.stdout:2070 write 0x6421a00 thru 0x64314eb (0xfaec bytes) 2024-09-15T00:09:40.478 INFO:teuthology.orchestra.run.smithi092.stdout:2074 read 0xcf61a00 thru 0xcf67515 (0x5b16 bytes) 2024-09-15T00:09:40.478 INFO:teuthology.orchestra.run.smithi092.stdout:2076 read 0x9e86000 thru 0x9e86715 (0x716 bytes) 2024-09-15T00:09:40.478 INFO:teuthology.orchestra.run.smithi092.stdout:2079 read 0x12b9000 thru 0x12c2ee2 (0x9ee3 bytes) 2024-09-15T00:09:40.481 INFO:teuthology.orchestra.run.smithi092.stdout:2080 read 0xbc2d800 thru 0xbc39b0b (0xc30c bytes) 2024-09-15T00:09:40.481 INFO:teuthology.orchestra.run.smithi092.stdout:2084 write 0x50400 thru 0x5193d (0x153e bytes) 2024-09-15T00:09:40.492 INFO:teuthology.orchestra.run.smithi092.stdout:2085 write 0xe2dec00 thru 0xe2e456b (0x596c bytes) 2024-09-15T00:09:40.495 INFO:teuthology.orchestra.run.smithi092.stdout:2086 punch from 0x883c200 to 0x8847600, (0xb400 bytes) 2024-09-15T00:09:40.495 INFO:teuthology.orchestra.run.smithi092.stdout:2092 write 0x1a0ac00 thru 0x1a13583 (0x8984 bytes) 2024-09-15T00:09:40.498 INFO:teuthology.orchestra.run.smithi092.stdout:2093 punch from 0x66c8000 to 0x66cbc00, (0x3c00 bytes) 2024-09-15T00:09:40.498 INFO:teuthology.orchestra.run.smithi092.stdout:2094 write 0x1047a00 thru 0x1048823 (0xe24 bytes) 2024-09-15T00:09:40.498 INFO:teuthology.orchestra.run.smithi092.stdout:2095 trunc from 0xe2e4600 to 0xc33fc00 2024-09-15T00:09:40.505 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:40.503+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:40.511 INFO:teuthology.orchestra.run.smithi092.stdout:2097 write 0x7511600 thru 0x751870e (0x710f bytes) 2024-09-15T00:09:40.512 INFO:teuthology.orchestra.run.smithi092.stdout:2099 write 0x2202400 thru 0x2202c00 (0x801 bytes) 2024-09-15T00:09:40.513 INFO:teuthology.orchestra.run.smithi092.stdout:2100 punch from 0x5185400 to 0x5192000, (0xcc00 bytes) 2024-09-15T00:09:40.513 INFO:teuthology.orchestra.run.smithi092.stdout:2101 write 0x70b1600 thru 0x70b47ee (0x31ef bytes) 2024-09-15T00:09:40.513 INFO:teuthology.orchestra.run.smithi092.stdout:2102 write 0xaf20200 thru 0xaf2cc72 (0xca73 bytes) 2024-09-15T00:09:40.513 INFO:teuthology.orchestra.run.smithi092.stdout:2105 write 0x520b400 thru 0x5219070 (0xdc71 bytes) 2024-09-15T00:09:40.525 INFO:teuthology.orchestra.run.smithi092.stdout:2106 trunc from 0xc33fc00 to 0x2254400 2024-09-15T00:09:40.546 INFO:teuthology.orchestra.run.smithi092.stdout:2109 read 0x1b02000 thru 0x1b0b941 (0x9942 bytes) 2024-09-15T00:09:40.548 INFO:teuthology.orchestra.run.smithi092.stdout:2110 punch from 0x1f02000 to 0x1f03200, (0x1200 bytes) 2024-09-15T00:09:40.549 INFO:teuthology.orchestra.run.smithi092.stdout:2112 punch from 0xb5d800 to 0xb69e00, (0xc600 bytes) 2024-09-15T00:09:40.554 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:40.554 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:40.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:40.570 INFO:teuthology.orchestra.run.smithi092.stdout:2113 write 0xc2bb800 thru 0xc2c6e5d (0xb65e bytes) 2024-09-15T00:09:40.571 INFO:teuthology.orchestra.run.smithi092.stdout:2118 write 0x60ea000 thru 0x60f3cb4 (0x9cb5 bytes) 2024-09-15T00:09:40.575 INFO:teuthology.orchestra.run.smithi092.stdout:2119 trunc from 0xc2c7000 to 0x8611c00 2024-09-15T00:09:40.595 INFO:teuthology.orchestra.run.smithi092.stdout:2120 read 0x708ea00 thru 0x7093ba2 (0x51a3 bytes) 2024-09-15T00:09:40.597 INFO:teuthology.orchestra.run.smithi092.stdout:2122 punch from 0x2b94a00 to 0x2ba2e00, (0xe400 bytes) 2024-09-15T00:09:40.601 INFO:teuthology.orchestra.run.smithi092.stdout:2125 read 0x50bf400 thru 0x50cef96 (0xfb97 bytes) 2024-09-15T00:09:40.601 INFO:teuthology.orchestra.run.smithi092.stdout:2126 write 0x6f16800 thru 0x6f1e5f2 (0x7df3 bytes) 2024-09-15T00:09:40.601 INFO:teuthology.orchestra.run.smithi092.stdout:2128 punch from 0x803a400 to 0x8041600, (0x7200 bytes) 2024-09-15T00:09:40.605 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:40.603+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:40.614 INFO:teuthology.orchestra.run.smithi092.stdout:2129 write 0xc5cdc00 thru 0xc5db0d8 (0xd4d9 bytes) 2024-09-15T00:09:40.615 INFO:teuthology.orchestra.run.smithi092.stdout:2132 read 0x9de3800 thru 0x9df1628 (0xde29 bytes) 2024-09-15T00:09:40.615 INFO:teuthology.orchestra.run.smithi092.stdout:2133 punch from 0x6030800 to 0x6039a00, (0x9200 bytes) 2024-09-15T00:09:40.617 INFO:teuthology.orchestra.run.smithi092.stdout:2134 read 0xbb60200 thru 0xbb623b3 (0x21b4 bytes) 2024-09-15T00:09:40.617 INFO:teuthology.orchestra.run.smithi092.stdout:2141 read 0x7983000 thru 0x798afc6 (0x7fc7 bytes) 2024-09-15T00:09:40.624 INFO:teuthology.orchestra.run.smithi092.stdout:2143 trunc from 0xc5db200 to 0x63dbe00 2024-09-15T00:09:40.640 INFO:teuthology.orchestra.run.smithi092.stdout:2144 trunc from 0x63dbe00 to 0x4250400 2024-09-15T00:09:40.667 INFO:teuthology.orchestra.run.smithi092.stdout:2145 punch from 0x3596600 to 0x3599200, (0x2c00 bytes) 2024-09-15T00:09:40.668 INFO:teuthology.orchestra.run.smithi092.stdout:2146 read 0x400e800 thru 0x4013fa2 (0x57a3 bytes) 2024-09-15T00:09:40.669 INFO:teuthology.orchestra.run.smithi092.stdout:2148 write 0x5cfa00 thru 0x5db89b (0xbe9c bytes) 2024-09-15T00:09:40.670 INFO:teuthology.orchestra.run.smithi092.stdout:2152 read 0x40e4e00 thru 0x40f2e48 (0xe049 bytes) 2024-09-15T00:09:40.671 INFO:teuthology.orchestra.run.smithi092.stdout:2153 trunc from 0x4250400 to 0x49cb000 2024-09-15T00:09:40.691 INFO:teuthology.orchestra.run.smithi092.stdout:2154 trunc from 0x49cb000 to 0xd199a00 2024-09-15T00:09:40.700 INFO:teuthology.orchestra.run.smithi092.stdout:2155 punch from 0x95f9200 to 0x9608a00, (0xf800 bytes) 2024-09-15T00:09:40.702 INFO:teuthology.orchestra.run.smithi092.stdout:2156 punch from 0xc442c00 to 0xc452600, (0xfa00 bytes) 2024-09-15T00:09:40.702 INFO:teuthology.orchestra.run.smithi092.stdout:2158 write 0x1f0de00 thru 0x1f10dc2 (0x2fc3 bytes) 2024-09-15T00:09:40.706 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:40.704+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:40.714 INFO:teuthology.orchestra.run.smithi092.stdout:2159 write 0xe4c5400 thru 0xe4cd0c9 (0x7cca bytes) 2024-09-15T00:09:40.715 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:40.715 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:40.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:40.736 INFO:teuthology.orchestra.run.smithi092.stdout:2160 trunc from 0xe4cd200 to 0x2950c00 2024-09-15T00:09:40.754 INFO:teuthology.orchestra.run.smithi092.stdout:2162 read 0x2836a00 thru 0x2837fa4 (0x15a5 bytes) 2024-09-15T00:09:40.770 INFO:teuthology.orchestra.run.smithi092.stdout:2163 trunc from 0x2950c00 to 0xba0b600 2024-09-15T00:09:40.779 INFO:teuthology.orchestra.run.smithi092.stdout:2164 punch from 0x95f2a00 to 0x95fb200, (0x8800 bytes) 2024-09-15T00:09:40.787 INFO:teuthology.orchestra.run.smithi092.stdout:2171 write 0x2969a00 thru 0x2977b6e (0xe16f bytes) 2024-09-15T00:09:40.787 INFO:teuthology.orchestra.run.smithi092.stdout:2172 write 0x8fd1000 thru 0x8fd6f62 (0x5f63 bytes) 2024-09-15T00:09:40.802 INFO:teuthology.orchestra.run.smithi092.stdout:2174 write 0xe858e00 thru 0xe865a8a (0xcc8b bytes) 2024-09-15T00:09:40.807 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:40.805+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:40.809 INFO:teuthology.orchestra.run.smithi092.stdout:2176 read 0x5b90e00 thru 0x5b984d2 (0x76d3 bytes) 2024-09-15T00:09:40.809 INFO:teuthology.orchestra.run.smithi092.stdout:2179 read 0x4ed2200 thru 0x4ed9f91 (0x7d92 bytes) 2024-09-15T00:09:40.809 INFO:teuthology.orchestra.run.smithi092.stdout:2180 read 0x2696e00 thru 0x269c62a (0x582b bytes) 2024-09-15T00:09:40.810 INFO:teuthology.orchestra.run.smithi092.stdout:2182 write 0x1fdf800 thru 0x1fe74b5 (0x7cb6 bytes) 2024-09-15T00:09:40.810 INFO:teuthology.orchestra.run.smithi092.stdout:2183 write 0xa374200 thru 0xa38275a (0xe55b bytes) 2024-09-15T00:09:40.817 INFO:teuthology.orchestra.run.smithi092.stdout:2185 trunc from 0xe865c00 to 0x8e56600 2024-09-15T00:09:40.831 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:40.832 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:40.838 INFO:teuthology.orchestra.run.smithi092.stdout:2187 punch from 0x34d1600 to 0x34d5200, (0x3c00 bytes) 2024-09-15T00:09:40.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:40.853 INFO:teuthology.orchestra.run.smithi092.stdout:2188 trunc from 0x8e56600 to 0xe50e200 2024-09-15T00:09:40.862 INFO:teuthology.orchestra.run.smithi092.stdout:2189 write 0x9f77e00 thru 0x9f7eb88 (0x6d89 bytes) 2024-09-15T00:09:40.869 INFO:teuthology.orchestra.run.smithi092.stdout:2191 read 0x69abc00 thru 0x69ad4e7 (0x18e8 bytes) 2024-09-15T00:09:40.869 INFO:teuthology.orchestra.run.smithi092.stdout:2192 write 0x9fd4e00 thru 0x9fdbcfd (0x6efe bytes) 2024-09-15T00:09:40.870 INFO:teuthology.orchestra.run.smithi092.stdout:2194 clone 20 order 19 su 32768 sc 12 2024-09-15T00:09:40.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:40.906+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:40.944 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:40.945 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:40.953 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:40.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:09:41.008 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:41.006+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:41.031 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone19 from 0xe50e200 (overlap 0x1bd0c00) to 0xde8600 2024-09-15T00:09:41.063 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:41.063 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:41.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:41.110 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:41.108+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:41.180 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:41.180 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:41.188 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:09:41.211 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:41.209+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:41.299 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:41.299 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:41.308 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:09:41.311 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:41.309+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:41.377 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-15T00:09:41.411 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:09:41.409+0000 7f57d4a60640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31785) UID: 0 2024-09-15T00:09:41.416 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:41.416 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:41.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:09:41.433 INFO:teuthology.orchestra.run.smithi092.stdout:2197 write 0xb978600 thru 0xb97d641 (0x5042 bytes) 2024-09-15T00:09:41.435 INFO:teuthology.orchestra.run.smithi092.stdout:2198 read 0xf4f400 thru 0xf5377b (0x437c bytes) 2024-09-15T00:09:41.440 INFO:teuthology.orchestra.run.smithi092.stdout:2202 punch from 0x9abbe00 to 0x9ac1e00, (0x6000 bytes) 2024-09-15T00:09:41.453 INFO:teuthology.orchestra.run.smithi092.stdout:2204 punch from 0x8c31c00 to 0x8c3ce00, (0xb200 bytes) 2024-09-15T00:09:41.459 INFO:teuthology.orchestra.run.smithi092.stdout:2208 read 0x301c200 thru 0x3022763 (0x6564 bytes) 2024-09-15T00:09:41.460 INFO:teuthology.orchestra.run.smithi092.stdout:2209 write 0x42b4800 thru 0x42bc27f (0x7a80 bytes) 2024-09-15T00:09:41.460 INFO:teuthology.orchestra.run.smithi092.stdout:2211 punch from 0x46dea00 to 0x46e2600, (0x3c00 bytes) 2024-09-15T00:09:41.470 INFO:teuthology.orchestra.run.smithi092.stdout:2212 write 0x858ce00 thru 0x8593ab2 (0x6cb3 bytes) 2024-09-15T00:09:41.470 INFO:teuthology.orchestra.run.smithi092.stdout:2215 write 0x676e200 thru 0x677a34c (0xc14d bytes) 2024-09-15T00:09:41.470 INFO:teuthology.orchestra.run.smithi092.stdout:2216 write 0x230fe00 thru 0x23145cc (0x47cd bytes) 2024-09-15T00:09:41.471 INFO:teuthology.orchestra.run.smithi092.stdout:2217 read 0x3945600 thru 0x394a53a (0x4f3b bytes) 2024-09-15T00:09:41.477 INFO:teuthology.orchestra.run.smithi092.stdout:2220 trunc from 0xe50e200 to 0x912fe00 2024-09-15T00:09:41.512 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:41.511+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:41.538 INFO:teuthology.orchestra.run.smithi092.stdout:2221 punch from 0x901a00 to 0x90aa00, (0x9000 bytes) 2024-09-15T00:09:41.571 INFO:teuthology.orchestra.run.smithi092.stdout:2227 write 0xab4b000 thru 0xab55237 (0xa238 bytes) 2024-09-15T00:09:41.571 INFO:teuthology.orchestra.run.smithi092.stdout:2229 read 0x8644200 thru 0x8647629 (0x342a bytes) 2024-09-15T00:09:41.571 INFO:teuthology.orchestra.run.smithi092.stdout:2230 punch from 0x230f800 to 0x2319a00, (0xa200 bytes) 2024-09-15T00:09:41.585 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:41.585 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:41.588 INFO:teuthology.orchestra.run.smithi092.stdout:2233 write 0xb073c00 thru 0xb08096c (0xcd6d bytes) 2024-09-15T00:09:41.589 INFO:teuthology.orchestra.run.smithi092.stdout:2234 read 0x58a6000 thru 0x58b00ff (0xa100 bytes) 2024-09-15T00:09:41.589 INFO:teuthology.orchestra.run.smithi092.stdout:2238 write 0x4001600 thru 0x400c91b (0xb31c bytes) 2024-09-15T00:09:41.589 INFO:teuthology.orchestra.run.smithi092.stdout:2239 read 0x4260600 thru 0x426cfa4 (0xc9a5 bytes) 2024-09-15T00:09:41.589 INFO:teuthology.orchestra.run.smithi092.stdout:2241 write 0x3943200 thru 0x394be8d (0x8c8e bytes) 2024-09-15T00:09:41.590 INFO:teuthology.orchestra.run.smithi092.stdout:2244 write 0x2743800 thru 0x275008c (0xc88d bytes) 2024-09-15T00:09:41.590 INFO:teuthology.orchestra.run.smithi092.stdout:2245 write 0x8523c00 thru 0x8528cd8 (0x50d9 bytes) 2024-09-15T00:09:41.593 INFO:teuthology.orchestra.run.smithi092.stdout:2247 punch from 0xd8de00 to 0xd8f600, (0x1800 bytes) 2024-09-15T00:09:41.595 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:41.620 INFO:teuthology.orchestra.run.smithi092.stdout:2248 write 0xb255000 thru 0xb25844d (0x344e bytes) 2024-09-15T00:09:41.621 INFO:teuthology.orchestra.run.smithi092.stdout:2250 write 0x4529a00 thru 0x45356da (0xbcdb bytes) 2024-09-15T00:09:41.621 INFO:teuthology.orchestra.run.smithi092.stdout:2252 read 0x94dd600 thru 0x94e32d8 (0x5cd9 bytes) 2024-09-15T00:09:41.622 INFO:teuthology.orchestra.run.smithi092.stdout:2253 read 0x6daae00 thru 0x6db9906 (0xeb07 bytes) 2024-09-15T00:09:41.622 INFO:teuthology.orchestra.run.smithi092.stdout:2255 write 0xa17fe00 thru 0xa1806cd (0x8ce bytes) 2024-09-15T00:09:41.637 INFO:teuthology.orchestra.run.smithi092.stdout:2256 write 0xbd3e400 thru 0xbd44afb (0x66fc bytes) 2024-09-15T00:09:41.637 INFO:teuthology.orchestra.run.smithi092.stdout:2257 read 0x74bc400 thru 0x74c418c (0x7d8d bytes) 2024-09-15T00:09:41.640 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:41.638+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:41.646 INFO:teuthology.orchestra.run.smithi092.stdout:2258 trunc from 0xbd44c00 to 0x4471800 2024-09-15T00:09:41.687 INFO:teuthology.orchestra.run.smithi092.stdout:2259 read 0x712e00 thru 0x71a050 (0x7251 bytes) 2024-09-15T00:09:41.693 INFO:teuthology.orchestra.run.smithi092.stdout:2261 read 0x207c000 thru 0x207ee56 (0x2e57 bytes) 2024-09-15T00:09:41.708 INFO:teuthology.orchestra.run.smithi092.stdout:2263 write 0x8c0b000 thru 0x8c198e4 (0xe8e5 bytes) 2024-09-15T00:09:41.710 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:41.710 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:41.714 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:41.712+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:41.715 INFO:teuthology.orchestra.run.smithi092.stdout:2264 punch from 0x5a3c800 to 0x5a48a00, (0xc200 bytes) 2024-09-15T00:09:41.716 INFO:teuthology.orchestra.run.smithi092.stdout:2268 punch from 0x3443800 to 0x3444400, (0xc00 bytes) 2024-09-15T00:09:41.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:41.736 INFO:teuthology.orchestra.run.smithi092.stdout:2269 write 0xc991600 thru 0xc9926ae (0x10af bytes) 2024-09-15T00:09:41.743 INFO:teuthology.orchestra.run.smithi092.stdout:2273 write 0x28fc200 thru 0x2906f05 (0xad06 bytes) 2024-09-15T00:09:41.743 INFO:teuthology.orchestra.run.smithi092.stdout:2274 punch from 0xa69d200 to 0xa69d800, (0x600 bytes) 2024-09-15T00:09:41.745 INFO:teuthology.orchestra.run.smithi092.stdout:2275 trunc from 0xc992800 to 0xc637a00 2024-09-15T00:09:41.777 INFO:teuthology.orchestra.run.smithi092.stdout:2277 trunc from 0xc637a00 to 0x84b6e00 2024-09-15T00:09:41.802 INFO:teuthology.orchestra.run.smithi092.stdout:2278 read 0x3f24e00 thru 0x3f34ca4 (0xfea5 bytes) 2024-09-15T00:09:41.802 INFO:teuthology.orchestra.run.smithi092.stdout:2280 read 0x5c44c00 thru 0x5c4b95c (0x6d5d bytes) 2024-09-15T00:09:41.803 INFO:teuthology.orchestra.run.smithi092.stdout:2281 read 0x5b65200 thru 0x5b6f2f0 (0xa0f1 bytes) 2024-09-15T00:09:41.803 INFO:teuthology.orchestra.run.smithi092.stdout:2282 write 0x3ea1c00 thru 0x3eac542 (0xa943 bytes) 2024-09-15T00:09:41.803 INFO:teuthology.orchestra.run.smithi092.stdout:2286 read 0x10fe600 thru 0x1101b2d (0x352e bytes) 2024-09-15T00:09:41.811 INFO:teuthology.orchestra.run.smithi092.stdout:2288 trunc from 0x84b6e00 to 0x227aa00 2024-09-15T00:09:41.815 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:41.814+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:41.870 INFO:teuthology.orchestra.run.smithi092.stdout:2291 punch from 0x3fec00 to 0x40ba00, (0xce00 bytes) 2024-09-15T00:09:41.870 INFO:teuthology.orchestra.run.smithi092.stdout:2295 read 0x1f11000 thru 0x1f18095 (0x7096 bytes) 2024-09-15T00:09:41.872 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:41.873 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:41.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:41.897 INFO:teuthology.orchestra.run.smithi092.stdout:2296 write 0x2555400 thru 0x255d7d2 (0x83d3 bytes) 2024-09-15T00:09:41.897 INFO:teuthology.orchestra.run.smithi092.stdout:2297 read 0x147200 thru 0x15545b (0xe25c bytes) 2024-09-15T00:09:41.898 INFO:teuthology.orchestra.run.smithi092.stdout:2298 read 0x2d400 thru 0x376d8 (0xa2d9 bytes) 2024-09-15T00:09:41.898 INFO:teuthology.orchestra.run.smithi092.stdout:2299 punch from 0xb7d400 to 0xb80000, (0x2c00 bytes) 2024-09-15T00:09:41.899 INFO:teuthology.orchestra.run.smithi092.stdout:2301 punch from 0x176e400 to 0x177de00, (0xfa00 bytes) 2024-09-15T00:09:41.899 INFO:teuthology.orchestra.run.smithi092.stdout:2302 write 0xe68600 thru 0xe6e595 (0x5f96 bytes) 2024-09-15T00:09:41.900 INFO:teuthology.orchestra.run.smithi092.stdout:2303 write 0xfd9200 thru 0xfe8a05 (0xf806 bytes) 2024-09-15T00:09:41.916 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:41.914+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:41.936 INFO:teuthology.orchestra.run.smithi092.stdout:2306 write 0x9699c00 thru 0x969cb6c (0x2f6d bytes) 2024-09-15T00:09:41.939 INFO:teuthology.orchestra.run.smithi092.stdout:2307 trunc from 0x969cc00 to 0x7674800 2024-09-15T00:09:41.955 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:09:41.982 INFO:teuthology.orchestra.run.smithi092.stdout:2312 write 0x9ec1000 thru 0x9ec7a0e (0x6a0f bytes) 2024-09-15T00:09:41.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:41.997 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:41.997 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:42.005 INFO:teuthology.orchestra.run.smithi092.stdout:2313 write 0xcf39800 thru 0xcf3ca82 (0x3283 bytes) 2024-09-15T00:09:42.005 INFO:teuthology.orchestra.run.smithi092.stdout:2314 punch from 0x95d9a00 to 0x95de600, (0x4c00 bytes) 2024-09-15T00:09:42.005 INFO:teuthology.orchestra.run.smithi092.stdout:2315 write 0x575d800 thru 0x57634d9 (0x5cda bytes) 2024-09-15T00:09:42.005 INFO:teuthology.orchestra.run.smithi092.stdout:2316 trunc from 0xcf3cc00 to 0xce86600 2024-09-15T00:09:42.006 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:42.017 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:42.015+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:42.028 INFO:teuthology.orchestra.run.smithi092.stdout:2317 read 0x87c1200 thru 0x87c8b6a (0x796b bytes) 2024-09-15T00:09:42.038 INFO:teuthology.orchestra.run.smithi092.stdout:2318 trunc from 0xce86600 to 0x4755e00 2024-09-15T00:09:42.059 INFO:teuthology.orchestra.run.smithi092.stdout:2319 write 0x4f5000 thru 0x4ff07c (0xa07d bytes) 2024-09-15T00:09:42.080 INFO:teuthology.orchestra.run.smithi092.stdout:2320 write 0x4cf8200 thru 0x4d02f49 (0xad4a bytes) 2024-09-15T00:09:42.083 INFO:teuthology.orchestra.run.smithi092.stdout:2323 trunc from 0x4d03000 to 0x6203600 2024-09-15T00:09:42.095 INFO:teuthology.orchestra.run.smithi092.stdout:2328 punch from 0x38c8400 to 0x38c8e00, (0xa00 bytes) 2024-09-15T00:09:42.097 INFO:teuthology.orchestra.run.smithi092.stdout:2333 read 0x5bcc000 thru 0x5bd0574 (0x4575 bytes) 2024-09-15T00:09:42.105 INFO:teuthology.orchestra.run.smithi092.stdout:2340 trunc from 0x6203600 to 0xcccec00 2024-09-15T00:09:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:09:42.114 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:09:42.117 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:42.115+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:42.122 INFO:teuthology.orchestra.run.smithi092.stdout:2341 trunc from 0xcccec00 to 0x6904e00 2024-09-15T00:09:42.131 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:42.131 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:42.133 INFO:teuthology.orchestra.run.smithi092.stdout:2342 read 0x604fe00 thru 0x6050df7 (0xff8 bytes) 2024-09-15T00:09:42.133 INFO:teuthology.orchestra.run.smithi092.stdout:2344 punch from 0x336aa00 to 0x337a000, (0xf600 bytes) 2024-09-15T00:09:42.134 INFO:teuthology.orchestra.run.smithi092.stdout:2346 punch from 0x2db3c00 to 0x2db6600, (0x2a00 bytes) 2024-09-15T00:09:42.135 INFO:teuthology.orchestra.run.smithi092.stdout:2347 read 0x2fad000 thru 0x2fb6154 (0x9155 bytes) 2024-09-15T00:09:42.138 INFO:teuthology.orchestra.run.smithi092.stdout:2348 trunc from 0x6904e00 to 0x3f63800 2024-09-15T00:09:42.149 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:42.149 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:42.161 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:42.161 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:42.161 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:42.165 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:42.166 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:42.170 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:42.171 INFO:teuthology.orchestra.run.smithi092.stdout:2350 write 0x41cac00 thru 0x41d7195 (0xc596 bytes) 2024-09-15T00:09:42.171 INFO:teuthology.orchestra.run.smithi092.stdout:2351 punch from 0x3796e00 to 0x37a5400, (0xe600 bytes) 2024-09-15T00:09:42.184 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:42.185 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:42.192 INFO:teuthology.orchestra.run.smithi092.stdout:2353 write 0x766b000 thru 0x7670ffb (0x5ffc bytes) 2024-09-15T00:09:42.193 INFO:teuthology.orchestra.run.smithi092.stdout:2354 punch from 0x70bc600 to 0x70bec00, (0x2600 bytes) 2024-09-15T00:09:42.193 INFO:teuthology.orchestra.run.smithi092.stdout:2355 read 0x7428200 thru 0x74300b0 (0x7eb1 bytes) 2024-09-15T00:09:42.193 INFO:teuthology.orchestra.run.smithi092.stdout:2359 read 0x3978a00 thru 0x397fb76 (0x7177 bytes) 2024-09-15T00:09:42.203 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:42.211 INFO:teuthology.orchestra.run.smithi092.stdout:2361 write 0x92c7600 thru 0x92ca596 (0x2f97 bytes) 2024-09-15T00:09:42.214 INFO:teuthology.orchestra.run.smithi092.stdout:2364 trunc from 0x92ca600 to 0xb1ad000 2024-09-15T00:09:42.218 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:42.216+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:42.233 INFO:teuthology.orchestra.run.smithi092.stdout:2365 write 0xa8d4c00 thru 0xa8dbf7f (0x7380 bytes) 2024-09-15T00:09:42.245 INFO:teuthology.orchestra.run.smithi092.stdout:2366 trunc from 0xb1ad000 to 0x1117e00 2024-09-15T00:09:42.285 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:42.285 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:42.285 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:42.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:42.319 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:42.317+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:42.325 INFO:teuthology.orchestra.run.smithi092.stdout:2367 trunc from 0x1117e00 to 0x63c2400 2024-09-15T00:09:42.339 INFO:teuthology.orchestra.run.smithi092.stdout:2371 punch from 0x4b17e00 to 0x4b22200, (0xa400 bytes) 2024-09-15T00:09:42.339 INFO:teuthology.orchestra.run.smithi092.stdout:2374 read 0x254fc00 thru 0x2555623 (0x5a24 bytes) 2024-09-15T00:09:42.360 INFO:teuthology.orchestra.run.smithi092.stdout:2377 write 0xc348c00 thru 0xc355c32 (0xd033 bytes) 2024-09-15T00:09:42.360 INFO:teuthology.orchestra.run.smithi092.stdout:2379 write 0x82dba00 thru 0x82eb1b7 (0xf7b8 bytes) 2024-09-15T00:09:42.372 INFO:teuthology.orchestra.run.smithi092.stdout:2381 trunc from 0xc355e00 to 0x1beae00 2024-09-15T00:09:42.402 INFO:teuthology.orchestra.run.smithi092.stdout:2382 punch from 0x4ed000 to 0x4f8c00, (0xbc00 bytes) 2024-09-15T00:09:42.402 INFO:teuthology.orchestra.run.smithi092.stdout:2383 read 0x1839e00 thru 0x183be0a (0x200b bytes) 2024-09-15T00:09:42.402 INFO:teuthology.orchestra.run.smithi092.stdout:2384 read 0x574c00 thru 0x57795f (0x2d60 bytes) 2024-09-15T00:09:42.404 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.376510978000001, 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.405 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:42.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:42.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:09:42.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:09:42.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.376489060000001, 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:09:42.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.414 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.376467787999999, 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.415 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.419 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.374726528, 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:09:42.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.374704621999999, 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:09:42.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.428 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.374682529999999, 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.429 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:09:42.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:09:42.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:09:42.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:09:42.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:09:42.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.436 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.436 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.436 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.374650791000001, 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:09:42.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.374352588000001, 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:09:42.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:09:42.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.374318883000001, 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.446 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:42.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:09:42.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:09:42.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:09:42.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.370016809999999, 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.451 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:42.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:09:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.369691939999999, 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:09:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:09:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:09:42.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:42.460 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.369281535000001, 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:09:42.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.369226632, 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:09:42.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.369194597, 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:09:42.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:09:42.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.367716766999999, 2024-09-15T00:09:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:42.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:09:42.478 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.367695052, 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.479 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:09:42.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.483 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.367673572999999, 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:09:42.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.365660469, 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:09:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:42.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:09:42.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.365640828, 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.493 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:09:42.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.36559428, 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.498 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:42.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:09:42.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:42.502 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:42.504 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:42.502+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:42.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:42.505 INFO:teuthology.orchestra.run.smithi092.stdout:2385 write 0x274600 thru 0x281b36 (0xd537 bytes) 2024-09-15T00:09:42.505 INFO:teuthology.orchestra.run.smithi092.stdout:2389 write 0x7866000 thru 0x7872788 (0xc789 bytes) 2024-09-15T00:09:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:2392 read 0x1596a00 thru 0x1596e00 (0x401 bytes) 2024-09-15T00:09:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:2394 punch from 0x1d40400 to 0x1d4d200, (0xce00 bytes) 2024-09-15T00:09:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:2395 trunc from 0x7872800 to 0x4bc7c00 2024-09-15T00:09:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:2396 read 0x4974c00 thru 0x497acf8 (0x60f9 bytes) 2024-09-15T00:09:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:2398 trunc from 0x4bc7c00 to 0x9161200 2024-09-15T00:09:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:2399 write 0xb4e0800 thru 0xb4e3870 (0x3071 bytes) 2024-09-15T00:09:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:2400 write 0xc642000 thru 0xc6461ad (0x41ae bytes) 2024-09-15T00:09:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:2402 read 0x7b84600 thru 0x7b94264 (0xfc65 bytes) 2024-09-15T00:09:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:2403 write 0x1022a00 thru 0x1030858 (0xde59 bytes) 2024-09-15T00:09:42.516 INFO:teuthology.orchestra.run.smithi092.stdout:2404 write 0xdcc4800 thru 0xdcd3e2f (0xf630 bytes) 2024-09-15T00:09:42.521 INFO:teuthology.orchestra.run.smithi092.stdout:2408 punch from 0xcb87800 to 0xcb91c00, (0xa400 bytes) 2024-09-15T00:09:42.523 INFO:teuthology.orchestra.run.smithi092.stdout:2411 read 0x2e28400 thru 0x2e2d289 (0x4e8a bytes) 2024-09-15T00:09:42.529 INFO:teuthology.orchestra.run.smithi092.stdout:2413 trunc from 0xdcd4000 to 0x8dc5200 2024-09-15T00:09:42.553 INFO:teuthology.orchestra.run.smithi092.stdout:2414 punch from 0x47c9400 to 0x47d3a00, (0xa600 bytes) 2024-09-15T00:09:42.573 INFO:teuthology.orchestra.run.smithi092.stdout:2415 write 0x9313400 thru 0x9316d01 (0x3902 bytes) 2024-09-15T00:09:42.578 INFO:teuthology.orchestra.run.smithi092.stdout:2416 read 0x3b84c00 thru 0x3b8abc4 (0x5fc5 bytes) 2024-09-15T00:09:42.578 INFO:teuthology.orchestra.run.smithi092.stdout:2421 read 0x5336c00 thru 0x533e5b0 (0x79b1 bytes) 2024-09-15T00:09:42.589 INFO:teuthology.orchestra.run.smithi092.stdout:2423 trunc from 0x9316e00 to 0x1a5e00 2024-09-15T00:09:42.604 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:42.602+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:42.631 INFO:teuthology.orchestra.run.smithi092.stdout:2424 punch from 0x173400 to 0x182400, (0xf000 bytes) 2024-09-15T00:09:42.634 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:42.634 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:42.634 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:42.634 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:42.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:42.658 INFO:teuthology.orchestra.run.smithi092.stdout:2425 write 0xbf96400 thru 0xbf9b3d0 (0x4fd1 bytes) 2024-09-15T00:09:42.665 INFO:teuthology.orchestra.run.smithi092.stdout:2427 read 0x451ec00 thru 0x452a121 (0xb522 bytes) 2024-09-15T00:09:42.665 INFO:teuthology.orchestra.run.smithi092.stdout:2434 read 0x9089a00 thru 0x908ce02 (0x3403 bytes) 2024-09-15T00:09:42.665 INFO:teuthology.orchestra.run.smithi092.stdout:2435 punch from 0x33bb000 to 0x33c8200, (0xd200 bytes) 2024-09-15T00:09:42.667 INFO:teuthology.orchestra.run.smithi092.stdout:2436 write 0x4ebc400 thru 0x4ec909d (0xcc9e bytes) 2024-09-15T00:09:42.667 INFO:teuthology.orchestra.run.smithi092.stdout:2437 write 0x8628000 thru 0x862aa8a (0x2a8b bytes) 2024-09-15T00:09:42.667 INFO:teuthology.orchestra.run.smithi092.stdout:2442 punch from 0x5eee800 to 0x5efa800, (0xc000 bytes) 2024-09-15T00:09:42.670 INFO:teuthology.orchestra.run.smithi092.stdout:2444 trunc from 0xbf9b400 to 0x96fea00 2024-09-15T00:09:42.695 INFO:teuthology.orchestra.run.smithi092.stdout:2445 read 0x8516e00 thru 0x851f811 (0x8a12 bytes) 2024-09-15T00:09:42.695 INFO:teuthology.orchestra.run.smithi092.stdout:2447 punch from 0x7ca1600 to 0x7ca8c00, (0x7600 bytes) 2024-09-15T00:09:42.697 INFO:teuthology.orchestra.run.smithi092.stdout:2448 write 0x89fa000 thru 0x8a02416 (0x8417 bytes) 2024-09-15T00:09:42.699 INFO:teuthology.orchestra.run.smithi092.stdout:2449 trunc from 0x96fea00 to 0x7e02200 2024-09-15T00:09:42.723 INFO:teuthology.orchestra.run.smithi092.stdout:2450 write 0x1ed7400 thru 0x1ed9ad4 (0x26d5 bytes) 2024-09-15T00:09:42.724 INFO:teuthology.orchestra.run.smithi092.stdout:2455 read 0x65e7400 thru 0x65f1f6c (0xab6d bytes) 2024-09-15T00:09:42.724 INFO:teuthology.orchestra.run.smithi092.stdout:2456 read 0x2f1ea00 thru 0x2f26b59 (0x815a bytes) 2024-09-15T00:09:42.728 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:42.727+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:42.743 INFO:teuthology.orchestra.run.smithi092.stdout:2458 write 0xa80c800 thru 0xa80d706 (0xf07 bytes) 2024-09-15T00:09:42.751 INFO:teuthology.orchestra.run.smithi092.stdout:2461 punch from 0x4ead200 to 0x4eb2000, (0x4e00 bytes) 2024-09-15T00:09:42.752 INFO:teuthology.orchestra.run.smithi092.stdout:2463 read 0x3414c00 thru 0x3422a58 (0xde59 bytes) 2024-09-15T00:09:42.753 INFO:teuthology.orchestra.run.smithi092.stdout:2464 read 0x43f6200 thru 0x43fa0a1 (0x3ea2 bytes) 2024-09-15T00:09:42.753 INFO:teuthology.orchestra.run.smithi092.stdout:2466 read 0xa114000 thru 0xa11ae24 (0x6e25 bytes) 2024-09-15T00:09:42.753 INFO:teuthology.orchestra.run.smithi092.stdout:2467 punch from 0x431de00 to 0x4328000, (0xa200 bytes) 2024-09-15T00:09:42.757 INFO:teuthology.orchestra.run.smithi092.stdout:2468 trunc from 0xa80d800 to 0x6dcc400 2024-09-15T00:09:42.790 INFO:teuthology.orchestra.run.smithi092.stdout:2470 write 0xa5a4400 thru 0xa5a99d2 (0x55d3 bytes) 2024-09-15T00:09:42.791 INFO:teuthology.orchestra.run.smithi092.stdout:2471 punch from 0x9e25a00 to 0x9e30200, (0xa800 bytes) 2024-09-15T00:09:42.796 INFO:teuthology.orchestra.run.smithi092.stdout:2474 write 0x5385600 thru 0x538eccf (0x96d0 bytes) 2024-09-15T00:09:42.797 INFO:teuthology.orchestra.run.smithi092.stdout:2475 read 0x9e0c600 thru 0x9e1c3ae (0xfdaf bytes) 2024-09-15T00:09:42.797 INFO:teuthology.orchestra.run.smithi092.stdout:2476 punch from 0x4e75400 to 0x4e7d000, (0x7c00 bytes) 2024-09-15T00:09:42.797 INFO:teuthology.orchestra.run.smithi092.stdout:2477 punch from 0x78fa00 to 0x794800, (0x4e00 bytes) 2024-09-15T00:09:42.797 INFO:teuthology.orchestra.run.smithi092.stdout:2478 write 0x535f000 thru 0x536c1b5 (0xd1b6 bytes) 2024-09-15T00:09:42.798 INFO:teuthology.orchestra.run.smithi092.stdout:2480 read 0x8bd1c00 thru 0x8bd357e (0x197f bytes) 2024-09-15T00:09:42.798 INFO:teuthology.orchestra.run.smithi092.stdout:2485 write 0x3cf5000 thru 0x3d04b38 (0xfb39 bytes) 2024-09-15T00:09:42.798 INFO:teuthology.orchestra.run.smithi092.stdout:2486 read 0x559dc00 thru 0x55ad6cf (0xfad0 bytes) 2024-09-15T00:09:42.798 INFO:teuthology.orchestra.run.smithi092.stdout:2487 write 0x7f90a00 thru 0x7f9472b (0x3d2c bytes) 2024-09-15T00:09:42.806 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:42.804+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:42.808 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:42.808 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:42.808 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:42.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:42.820 INFO:teuthology.orchestra.run.smithi092.stdout:2488 write 0xddc8c00 thru 0xddd8001 (0xf402 bytes) 2024-09-15T00:09:42.828 INFO:teuthology.orchestra.run.smithi092.stdout:2489 read 0x926a800 thru 0x9278669 (0xde6a bytes) 2024-09-15T00:09:42.828 INFO:teuthology.orchestra.run.smithi092.stdout:2493 write 0x2fdaa00 thru 0x2fea2ff (0xf900 bytes) 2024-09-15T00:09:42.828 INFO:teuthology.orchestra.run.smithi092.stdout:2495 read 0x833e000 thru 0x8348744 (0xa745 bytes) 2024-09-15T00:09:42.828 INFO:teuthology.orchestra.run.smithi092.stdout:2497 write 0x5316200 thru 0x531af1b (0x4d1c bytes) 2024-09-15T00:09:42.829 INFO:teuthology.orchestra.run.smithi092.stdout:2498 read 0xeb800 thru 0xf3e36 (0x8637 bytes) 2024-09-15T00:09:42.829 INFO:teuthology.orchestra.run.smithi092.stdout:2502 write 0xb1af000 thru 0xb1b0521 (0x1522 bytes) 2024-09-15T00:09:42.835 INFO:teuthology.orchestra.run.smithi092.stdout:2503 trunc from 0xddd8200 to 0x8c54000 2024-09-15T00:09:42.861 INFO:teuthology.orchestra.run.smithi092.stdout:2505 punch from 0x84ab000 to 0x84ae000, (0x3000 bytes) 2024-09-15T00:09:42.875 INFO:teuthology.orchestra.run.smithi092.stdout:2507 trunc from 0x8c54000 to 0xc39000 2024-09-15T00:09:42.907 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:42.905+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:42.917 INFO:teuthology.orchestra.run.smithi092.stdout:2508 trunc from 0xc39000 to 0x9658400 2024-09-15T00:09:42.932 INFO:teuthology.orchestra.run.smithi092.stdout:2509 trunc from 0x9658400 to 0xd4bd600 2024-09-15T00:09:42.941 INFO:teuthology.orchestra.run.smithi092.stdout:2510 read 0x6bde200 thru 0x6be8390 (0xa191 bytes) 2024-09-15T00:09:42.942 INFO:teuthology.orchestra.run.smithi092.stdout:2511 read 0x59c7800 thru 0x59d5c15 (0xe416 bytes) 2024-09-15T00:09:42.942 INFO:teuthology.orchestra.run.smithi092.stdout:2512 punch from 0x6d52e00 to 0x6d60600, (0xd800 bytes) 2024-09-15T00:09:42.944 INFO:teuthology.orchestra.run.smithi092.stdout:2513 write 0x3ee7800 thru 0x3ef1c6d (0xa46e bytes) 2024-09-15T00:09:42.944 INFO:teuthology.orchestra.run.smithi092.stdout:2516 punch from 0x1bd9c00 to 0x1be6400, (0xc800 bytes) 2024-09-15T00:09:42.945 INFO:teuthology.orchestra.run.smithi092.stdout:2517 read 0x5fde000 thru 0x5fe1b0b (0x3b0c bytes) 2024-09-15T00:09:42.945 INFO:teuthology.orchestra.run.smithi092.stdout:2518 punch from 0xa3a0e00 to 0xa3ada00, (0xcc00 bytes) 2024-09-15T00:09:42.945 INFO:teuthology.orchestra.run.smithi092.stdout:2520 read 0x7ced600 thru 0x7cf4a20 (0x7421 bytes) 2024-09-15T00:09:42.945 INFO:teuthology.orchestra.run.smithi092.stdout:2522 clone 21 order 18 su 65536 sc 14 2024-09-15T00:09:42.957 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-15T00:09:42.957 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:09:42.958 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 4, 5] 2024-09-15T00:09:42.958 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T00:09:42.958 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:09:42.976 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4070 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622615803999999, 2024-09-15T00:09:42.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02702953, 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4070 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:42.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350999+0000", 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351003+0000", 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351007+0000", 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6820000000000001e-06 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351010+0000", 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7269999999999999e-06 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377929+0000", 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026919887999999999 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377969+0000", 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9909e-05 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378028+0000", 2024-09-15T00:09:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8578000000000001e-05 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4071 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622600924, 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027136008, 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:42.981 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4071 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:42.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351014+0000", 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351015+0000", 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7039999999999999e-06 2024-09-15T00:09:42.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:42.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351020+0000", 2024-09-15T00:09:42.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:09:42.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351023+0000", 2024-09-15T00:09:42.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378053+0000", 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027029610999999999 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378094+0000", 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1826999999999998e-05 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378149+0000", 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4815000000000003e-05 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4072 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622576245000001, 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027223404999999999, 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4072 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351040+0000", 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351046+0000", 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351051+0000", 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378171+0000", 2024-09-15T00:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027120590999999999 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378209+0000", 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7308999999999998e-05 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378261+0000", 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2753999999999997e-05 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4074 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622557350000001, 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027334509999999999, 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4074 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9200000000000002e-07 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351059+0000", 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.012e-06 2024-09-15T00:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351065+0000", 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2129999999999996e-06 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351070+0000", 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8470000000000003e-06 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378285+0000", 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027214449000000002 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378334+0000", 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378391+0000", 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7046000000000002e-05 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4075 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622538478999999, 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027546923000000001, 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4075 2024-09-15T00:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351078+0000", 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.905e-06 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351085+0000", 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351090+0000", 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378423+0000", 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027333051000000001 2024-09-15T00:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378523+0000", 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9959999999999998e-05 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378623+0000", 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9962000000000003e-05 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4077 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622515794999998, 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027660619000000001, 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4077 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351099+0000", 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351101+0000", 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351107+0000", 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7980000000000002e-06 2024-09-15T00:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351111+0000", 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378657+0000", 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027545463999999999 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378702+0000", 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5809999999999997e-05 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378759+0000", 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4078 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622487795000001, 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027808405000000001, 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4078 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351127+0000", 2024-09-15T00:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351129+0000", 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351135+0000", 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3820000000000001e-06 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351141+0000", 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2259999999999996e-06 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378785+0000", 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027644592999999999 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378855+0000", 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0276000000000003e-05 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378935+0000", 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9361000000000001e-05 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4079 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622466158000002, 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027913778, 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4079 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351150+0000", 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9010000000000001e-06 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351157+0000", 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-15T00:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351162+0000", 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378958+0000", 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027795786999999999 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379004+0000", 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6124000000000001e-05 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379062+0000", 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7630999999999998e-05 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4081 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622442477, 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016658, 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4081 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351174+0000", 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351181+0000", 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9070000000000001e-06 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351186+0000", 2024-09-15T00:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.651e-06 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379085+0000", 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027898757999999999 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379130+0000", 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5126000000000001e-05 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379188+0000", 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8548999999999998e-05 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4082 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622422423, 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028118457999999999, 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4082 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351194+0000", 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7519999999999999e-06 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351200+0000", 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.801e-06 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351205+0000", 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379213+0000", 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028007539000000001 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379254+0000", 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1817999999999998e-05 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379310+0000", 2024-09-15T00:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5850000000000002e-05 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4083 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622400703, 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028218838999999999, 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4083 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351214+0000", 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351216+0000", 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351222+0000", 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351226+0000", 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379333+0000", 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028106740000000002 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379376+0000", 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3581000000000002e-05 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379432+0000", 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5868000000000001e-05 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4085 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622372199000001, 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311823, 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4085 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0699999999999998e-07 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351244+0000", 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.782e-06 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351248+0000", 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6289999999999999e-06 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351250+0000", 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379457+0000", 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028206479 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379499+0000", 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1637000000000001e-05 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379554+0000", 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5248999999999999e-05 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4086 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622360358000002, 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02842368, 2024-09-15T00:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4086 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351259+0000", 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9520000000000002e-06 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351262+0000", 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351265+0000", 2024-09-15T00:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379576+0000", 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311228000000001 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379619+0000", 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2595000000000001e-05 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379678+0000", 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4087 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622346003000001, 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523903999999999, 2024-09-15T00:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4087 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351269+0000", 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351270+0000", 2024-09-15T00:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1090000000000001e-06 2024-09-15T00:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351273+0000", 2024-09-15T00:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9189999999999999e-06 2024-09-15T00:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351275+0000", 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379701+0000", 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028425813000000001 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379742+0000", 2024-09-15T00:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0769000000000002e-05 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379792+0000", 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4088 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622329869000001, 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028621284, 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4088 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351285+0000", 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351286+0000", 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.186e-06 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351289+0000", 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351292+0000", 2024-09-15T00:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379815+0000", 2024-09-15T00:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523211 2024-09-15T00:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379857+0000", 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1807999999999997e-05 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379906+0000", 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8594999999999998e-05 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4090 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622318784000001, 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028748587999999999, 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4090 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351296+0000", 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351297+0000", 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1060000000000001e-06 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351300+0000", 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9110000000000002e-06 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351302+0000", 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379947+0000", 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028645177000000001 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379991+0000", 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3680999999999997e-05 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380044+0000", 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2957000000000001e-05 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4091 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622309228999999, 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028864198000000001, 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4091 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351306+0000", 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-15T00:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351309+0000", 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351312+0000", 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380069+0000", 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028757933999999999 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380113+0000", 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3810000000000002e-05 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380169+0000", 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5887999999999998e-05 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4092 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622299870999999, 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028973819000000001, 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4092 2024-09-15T00:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351315+0000", 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9999999999999999e-07 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351316+0000", 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.95e-07 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351318+0000", 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8629999999999999e-06 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351321+0000", 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380195+0000", 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028873868 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380238+0000", 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2883999999999998e-05 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380288+0000", 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0096000000000003e-05 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4094 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622290251999999, 2024-09-15T00:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02907964, 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4094 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351325+0000", 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.054e-06 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351328+0000", 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351331+0000", 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4679999999999999e-06 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380312+0000", 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028981587999999999 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380355+0000", 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2688000000000002e-05 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380404+0000", 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8767999999999998e-05 2024-09-15T00:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4095 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.622280865, 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029185068000000002, 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4095 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351334+0000", 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.04e-07 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351335+0000", 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0300000000000001e-06 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351339+0000", 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-15T00:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351344+0000", 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0560000000000004e-06 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380428+0000", 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02908444 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380470+0000", 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1338000000000003e-05 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380518+0000", 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8724999999999999e-05 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:43.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:43.075 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:43.073+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:43.174 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:43.172+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:43.220 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:43.220 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:43.221 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:43.221 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:43.229 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:43.275 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:43.273+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:43.372 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:43.372 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:43.372 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:43.372 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:43.372 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:43.376 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:43.374+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:43.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:43.477 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:43.476+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:43.524 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:43.524 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:43.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1142 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.639341706, 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045190363999999997, 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1142 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882756+0000", 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882773+0000", 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6872999999999999e-05 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882780+0000", 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4240000000000004e-06 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882787+0000", 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8510000000000001e-06 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927799+0000", 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045011974000000003 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927843+0000", 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4270999999999997e-05 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927946+0000", 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102581 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.639293014, 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.529 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:43.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:09:43.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.639254384000001, 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:09:43.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:09:43.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.639228447000001, 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:09:43.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.542 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.639212379, 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:09:43.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:09:43.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.639071284, 2024-09-15T00:09:43.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:43.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:09:43.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:09:43.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.638299529000001, 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.552 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:09:43.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:09:43.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.638214498, 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:09:43.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.638179481999998, 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:09:43.561 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:09:43.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.638153879000001, 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.566 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:43.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:09:43.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.637979612999999, 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:09:43.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:09:43.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.575 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.637748473999999, 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:09:43.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1157 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.580 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.636967036000001, 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045184281, 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1157 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:43.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885132+0000", 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885139+0000", 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0350000000000004e-06 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885153+0000", 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3515e-05 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930177+0000", 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045024495999999997 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930249+0000", 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2072999999999999e-05 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930314+0000", 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4826999999999994e-05 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.63693653, 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.585 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:09:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:09:43.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:09:43.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.590 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.63685679, 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:09:43.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:09:43.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.636651765, 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:09:43.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.636440231999998, 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:09:43.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.633884224999999, 2024-09-15T00:09:43.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:43.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:09:43.608 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.631901118999998, 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.609 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:09:43.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.631881547999999, 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.614 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:09:43.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:43.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:43.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:09:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:09:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:43.618 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:43.618 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:43.620 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:43.618+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:43.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:43.721 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:43.718+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:43.731 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:43.731 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:43.731 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:43.731 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:43.740 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:43.825 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:43.823+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:43.887 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:43.888 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:43.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:43.922 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:43.921+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:43.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:09:44.024 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:44.023+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:44.047 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:44.047 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:44.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:44.047 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:44.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.047 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.047 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.01739141, 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.048 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:09:44.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:09:44.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.052 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.017371154999999, 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:09:44.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:09:44.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.017351409, 2024-09-15T00:09:44.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:44.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:09:44.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.017331458999999, 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.062 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:09:44.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:09:44.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.017303958999999, 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:09:44.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:09:44.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:44.071 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.017287118, 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:09:44.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.01727112, 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:09:44.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.017248132000001, 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.085 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.017222247999999, 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.090 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.017205820999999, 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.091 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:09:44.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.095 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.01719155, 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.096 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:09:44.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.017182233, 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:09:44.100 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:09:44.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:09:44.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.015513221000001, 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.105 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:09:44.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.015478158000001, 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:09:44.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:09:44.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.015453130999999, 2024-09-15T00:09:44.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:09:44.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:09:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.015428823000001, 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:44.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:09:44.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2674 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3768320~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.013282477000001, 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059850591000000002, 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2674 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.124 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031671+0000", 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031682+0000", 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1248e-05 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031693+0000", 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1395e-05 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031705+0000", 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2106000000000001e-05 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091352+0000", 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059646421999999998 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091441+0000", 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9174999999999999e-05 2024-09-15T00:09:44.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091521+0000", 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9636999999999997e-05 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.013236572, 2024-09-15T00:09:44.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:44.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:09:44.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.01321244, 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:09:44.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:09:44.137 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.013183068, 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:44.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:44.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:44.142 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:44.143 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:44.144 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:44.144 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:44.143+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:44.215 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:44.253 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:44.253 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:44.253 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:44.253 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:44.253 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:44.251+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:44.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:44.323 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:09:44.341 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:44.341 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:44.345 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:44.343+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:44.358 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:44.358 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:44.375 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:44.376 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:44.393 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:44.393 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:44.410 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:44.410 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:44.413 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:09:44.413 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:09:44.414 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:44.414 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:44.422 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:44.446 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:44.444+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:44.548 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:44.546+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:44.569 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:44.569 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:44.578 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:44.648 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:44.645+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:44.693 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:44.693 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:44.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:44.786 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:44.784+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:44.845 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:44.846 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:44.850 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:44.848+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:44.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:44.951 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:44.949+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:09:45.001 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:45.002 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:45.010 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:45.052 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:45.049+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:45.153 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:45.150+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:45.154 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:45.155 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:45.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:45.253 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:45.251+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:45.307 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:45.308 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:45.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:45.354 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:45.352+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:45.455 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:45.452+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:45.459 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:45.459 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:45.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:45.556 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:45.554+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:45.620 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:45.662 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:45.659+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:45.758 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:45.757+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:45.762 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:45.762 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:45.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:45.859 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:45.857+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:45.871 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:45.872 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:45.880 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:45.960 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:45.958+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:45.996 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:45.997 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:46.006 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:46.061 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:46.058+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:46.070 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone20 2024-09-15T00:09:46.114 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:46.114 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:46.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:46.167 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:46.165+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:46.230 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:46.230 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:46.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:46.263 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:46.260+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:46.364 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:46.362+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:46.387 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:46.387 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:46.396 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:46.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:46.465 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:46.462+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:46.508 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:46.508 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:46.518 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:46.520 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-15T00:09:46.550 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:09:46.566 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:46.564+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:46.571 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:46.589 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:46.589 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:46.606 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:46.606 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:46.623 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:46.633 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.604915686999998, 2024-09-15T00:09:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:09:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:09:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.604893769, 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:09:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.604872496999999, 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:09:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.603131236999999, 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:09:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.603109330999999, 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:09:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.603087239000001, 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:09:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.6030555, 2024-09-15T00:09:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:09:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:09:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.602757297, 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:09:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:09:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.602723592, 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:09:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.598421518999999, 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:09:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.598096648999999, 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:09:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:09:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.597686243999998, 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:09:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:09:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.597631341, 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:09:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.597599305999999, 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:09:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.596121476, 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:09:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:09:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:09:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.596099761000001, 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:09:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.596078282000001, 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:09:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.594065178000001, 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:09:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.594045537, 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:09:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:09:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:09:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.593998988999999, 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:09:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:09:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:46.742 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:46.744 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:46.744 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:46.741+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:46.744 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:46.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:46.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:09:46.747 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:09:46.748 INFO:teuthology.orchestra.run.smithi092.stdout:2523 write 0x4f38200 thru 0x4f3cbc2 (0x49c3 bytes) 2024-09-15T00:09:46.748 INFO:teuthology.orchestra.run.smithi092.stdout:2524 read 0xa2ed800 thru 0xa2fa307 (0xcb08 bytes) 2024-09-15T00:09:46.748 INFO:teuthology.orchestra.run.smithi092.stdout:2525 write 0x7aac600 thru 0x7ab6979 (0xa37a bytes) 2024-09-15T00:09:46.748 INFO:teuthology.orchestra.run.smithi092.stdout:2526 punch from 0x3161400 to 0x316ac00, (0x9800 bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2527 punch from 0xc6b8e00 to 0xc6b9e00, (0x1000 bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2530 read 0x2144400 thru 0x214f7b6 (0xb3b7 bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2531 write 0x3bee600 thru 0x3bf9582 (0xaf83 bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2532 read 0xc3d5600 thru 0xc3e2afe (0xd4ff bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2536 write 0x6c01e00 thru 0x6c108dd (0xeade bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2537 write 0x65d5e00 thru 0x65dc148 (0x6349 bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2539 trunc from 0xd4bd600 to 0x47bb000 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2540 read 0x7d1000 thru 0x7dd5a5 (0xc5a6 bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2543 write 0x1517c00 thru 0x1523898 (0xbc99 bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2546 punch from 0x25f0400 to 0x25f2400, (0x2000 bytes) 2024-09-15T00:09:46.749 INFO:teuthology.orchestra.run.smithi092.stdout:2548 write 0x1995200 thru 0x199e0b5 (0x8eb6 bytes) 2024-09-15T00:09:46.750 INFO:teuthology.orchestra.run.smithi092.stdout:2551 read 0x1ded000 thru 0x1dedc71 (0xc72 bytes) 2024-09-15T00:09:46.750 INFO:teuthology.orchestra.run.smithi092.stdout:2552 write 0x179fa00 thru 0x17a3e38 (0x4439 bytes) 2024-09-15T00:09:46.750 INFO:teuthology.orchestra.run.smithi092.stdout:2555 read 0x427a800 thru 0x427e976 (0x4177 bytes) 2024-09-15T00:09:46.750 INFO:teuthology.orchestra.run.smithi092.stdout:2556 trunc from 0x47bb000 to 0x156200 2024-09-15T00:09:46.794 INFO:teuthology.orchestra.run.smithi092.stdout:2557 trunc from 0x156200 to 0xdfba200 2024-09-15T00:09:46.809 INFO:teuthology.orchestra.run.smithi092.stdout:2559 read 0x5f47000 thru 0x5f50a4a (0x9a4b bytes) 2024-09-15T00:09:46.815 INFO:teuthology.orchestra.run.smithi092.stdout:2560 trunc from 0xdfba200 to 0x8ed8400 2024-09-15T00:09:46.825 INFO:teuthology.orchestra.run.smithi092.stdout:2562 read 0x53e0800 thru 0x53ed0c9 (0xc8ca bytes) 2024-09-15T00:09:46.826 INFO:teuthology.orchestra.run.smithi092.stdout:2564 write 0x1375a00 thru 0x13855a0 (0xfba1 bytes) 2024-09-15T00:09:46.827 INFO:teuthology.orchestra.run.smithi092.stdout:2565 read 0x297e200 thru 0x29896eb (0xb4ec bytes) 2024-09-15T00:09:46.843 INFO:teuthology.orchestra.run.smithi092.stdout:2568 write 0xd437a00 thru 0xd43a98d (0x2f8e bytes) 2024-09-15T00:09:46.844 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:46.842+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:46.850 INFO:teuthology.orchestra.run.smithi092.stdout:2569 read 0xd03ae00 thru 0xd03f820 (0x4a21 bytes) 2024-09-15T00:09:46.850 INFO:teuthology.orchestra.run.smithi092.stdout:2570 read 0xabbce00 thru 0xabc7e5a (0xb05b bytes) 2024-09-15T00:09:46.850 INFO:teuthology.orchestra.run.smithi092.stdout:2574 write 0x2757a00 thru 0x275db4e (0x614f bytes) 2024-09-15T00:09:46.851 INFO:teuthology.orchestra.run.smithi092.stdout:2575 write 0x69b1a00 thru 0x69c0676 (0xec77 bytes) 2024-09-15T00:09:46.851 INFO:teuthology.orchestra.run.smithi092.stdout:2576 read 0x165aa00 thru 0x16663bf (0xb9c0 bytes) 2024-09-15T00:09:46.851 INFO:teuthology.orchestra.run.smithi092.stdout:2579 write 0x2c5e800 thru 0x2c628b9 (0x40ba bytes) 2024-09-15T00:09:46.851 INFO:teuthology.orchestra.run.smithi092.stdout:2580 read 0xa8fdc00 thru 0xa909163 (0xb564 bytes) 2024-09-15T00:09:46.851 INFO:teuthology.orchestra.run.smithi092.stdout:2581 write 0x46de600 thru 0x46e60c6 (0x7ac7 bytes) 2024-09-15T00:09:46.855 INFO:teuthology.orchestra.run.smithi092.stdout:2582 trunc from 0xd43aa00 to 0x9ccbe00 2024-09-15T00:09:46.870 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:46.870 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:46.870 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:46.870 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:46.879 INFO:teuthology.orchestra.run.smithi092.stdout:2583 read 0x91fa800 thru 0x9207114 (0xc915 bytes) 2024-09-15T00:09:46.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:46.881 INFO:teuthology.orchestra.run.smithi092.stdout:2584 read 0x9b66200 thru 0x9b66260 (0x61 bytes) 2024-09-15T00:09:46.885 INFO:teuthology.orchestra.run.smithi092.stdout:2587 trunc from 0x9ccbe00 to 0xd719800 2024-09-15T00:09:46.894 INFO:teuthology.orchestra.run.smithi092.stdout:2588 punch from 0x63ef000 to 0x63fd000, (0xe000 bytes) 2024-09-15T00:09:46.897 INFO:teuthology.orchestra.run.smithi092.stdout:2589 read 0xaaf4c00 thru 0xab03d92 (0xf193 bytes) 2024-09-15T00:09:46.897 INFO:teuthology.orchestra.run.smithi092.stdout:2590 read 0xa524600 thru 0xa52ea0a (0xa40b bytes) 2024-09-15T00:09:46.897 INFO:teuthology.orchestra.run.smithi092.stdout:2595 write 0x63bca00 thru 0x63cb677 (0xec78 bytes) 2024-09-15T00:09:46.897 INFO:teuthology.orchestra.run.smithi092.stdout:2596 read 0x8143800 thru 0x814f4f4 (0xbcf5 bytes) 2024-09-15T00:09:46.897 INFO:teuthology.orchestra.run.smithi092.stdout:2598 read 0x3258c00 thru 0x325afc9 (0x23ca bytes) 2024-09-15T00:09:46.897 INFO:teuthology.orchestra.run.smithi092.stdout:2599 punch from 0x1c90a00 to 0x1c91600, (0xc00 bytes) 2024-09-15T00:09:46.898 INFO:teuthology.orchestra.run.smithi092.stdout:2601 punch from 0x2e52400 to 0x2e57400, (0x5000 bytes) 2024-09-15T00:09:46.899 INFO:teuthology.orchestra.run.smithi092.stdout:2602 read 0x1913a00 thru 0x191dce9 (0xa2ea bytes) 2024-09-15T00:09:46.899 INFO:teuthology.orchestra.run.smithi092.stdout:2603 read 0xd598200 thru 0xd59dd22 (0x5b23 bytes) 2024-09-15T00:09:46.899 INFO:teuthology.orchestra.run.smithi092.stdout:2605 write 0xc0e2200 thru 0xc0e93bb (0x71bc bytes) 2024-09-15T00:09:46.899 INFO:teuthology.orchestra.run.smithi092.stdout:2607 write 0x6dcbc00 thru 0x6dd272c (0x6b2d bytes) 2024-09-15T00:09:46.899 INFO:teuthology.orchestra.run.smithi092.stdout:2609 write 0x9ee6400 thru 0x9ef5c93 (0xf894 bytes) 2024-09-15T00:09:46.902 INFO:teuthology.orchestra.run.smithi092.stdout:2610 trunc from 0xd719800 to 0xa2f6600 2024-09-15T00:09:46.926 INFO:teuthology.orchestra.run.smithi092.stdout:2611 trunc from 0xa2f6600 to 0x901b800 2024-09-15T00:09:46.944 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:46.941+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:46.963 INFO:teuthology.orchestra.run.smithi092.stdout:2612 write 0xa473000 thru 0xa47b8ae (0x88af bytes) 2024-09-15T00:09:46.966 INFO:teuthology.orchestra.run.smithi092.stdout:2616 trunc from 0xa47ba00 to 0x7cf6400 2024-09-15T00:09:46.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:09:46.997 INFO:teuthology.orchestra.run.smithi092.stdout:2623 trunc from 0x7cf6400 to 0x1c4ae00 2024-09-15T00:09:46.998 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:46.999 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:46.999 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:46.999 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:46.999 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:47.010 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:47.017 INFO:teuthology.orchestra.run.smithi092.stdout:2625 read 0x119a200 thru 0x11a2728 (0x8529 bytes) 2024-09-15T00:09:47.018 INFO:teuthology.orchestra.run.smithi092.stdout:2626 read 0x4b9600 thru 0x4c4609 (0xb00a bytes) 2024-09-15T00:09:47.018 INFO:teuthology.orchestra.run.smithi092.stdout:2629 punch from 0x550a00 to 0x552800, (0x1e00 bytes) 2024-09-15T00:09:47.022 INFO:teuthology.orchestra.run.smithi092.stdout:2630 punch from 0xe62800 to 0xe70000, (0xd800 bytes) 2024-09-15T00:09:47.025 INFO:teuthology.orchestra.run.smithi092.stdout:2632 trunc from 0x1c4ae00 to 0x6447600 2024-09-15T00:09:47.049 INFO:teuthology.orchestra.run.smithi092.stdout:2636 write 0x7192800 thru 0x7195003 (0x2804 bytes) 2024-09-15T00:09:47.050 INFO:teuthology.orchestra.run.smithi092.stdout:2637 read 0x3b39c00 thru 0x3b3b3bd (0x17be bytes) 2024-09-15T00:09:47.050 INFO:teuthology.orchestra.run.smithi092.stdout:2640 read 0x425e400 thru 0x4260e93 (0x2a94 bytes) 2024-09-15T00:09:47.051 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:47.048+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:47.054 INFO:teuthology.orchestra.run.smithi092.stdout:2641 punch from 0x197e00 to 0x199200, (0x1400 bytes) 2024-09-15T00:09:47.055 INFO:teuthology.orchestra.run.smithi092.stdout:2642 read 0x3667e00 thru 0x367045a (0x865b bytes) 2024-09-15T00:09:47.055 INFO:teuthology.orchestra.run.smithi092.stdout:2644 punch from 0x15e800 to 0x16ae00, (0xc600 bytes) 2024-09-15T00:09:47.055 INFO:teuthology.orchestra.run.smithi092.stdout:2647 read 0x655f600 thru 0x656854d (0x8f4e bytes) 2024-09-15T00:09:47.055 INFO:teuthology.orchestra.run.smithi092.stdout:2648 read 0x2f0dc00 thru 0x2f164bc (0x88bd bytes) 2024-09-15T00:09:47.056 INFO:teuthology.orchestra.run.smithi092.stdout:2649 trunc from 0x7195200 to 0x3815e00 2024-09-15T00:09:47.069 INFO:teuthology.orchestra.run.smithi092.stdout:2651 punch from 0x299f400 to 0x29a5a00, (0x6600 bytes) 2024-09-15T00:09:47.075 INFO:teuthology.orchestra.run.smithi092.stdout:2652 write 0x2443c00 thru 0x244c376 (0x8777 bytes) 2024-09-15T00:09:47.075 INFO:teuthology.orchestra.run.smithi092.stdout:2653 read 0x2958600 thru 0x295fe5a (0x785b bytes) 2024-09-15T00:09:47.075 INFO:teuthology.orchestra.run.smithi092.stdout:2654 read 0x6e9200 thru 0x6f5562 (0xc363 bytes) 2024-09-15T00:09:47.076 INFO:teuthology.orchestra.run.smithi092.stdout:2656 trunc from 0x3815e00 to 0x291bc00 2024-09-15T00:09:47.093 INFO:teuthology.orchestra.run.smithi092.stdout:2657 punch from 0x6c2c00 to 0x6ce800, (0xbc00 bytes) 2024-09-15T00:09:47.098 INFO:teuthology.orchestra.run.smithi092.stdout:2659 write 0x26e8400 thru 0x26f11d2 (0x8dd3 bytes) 2024-09-15T00:09:47.099 INFO:teuthology.orchestra.run.smithi092.stdout:2660 read 0x8ba600 thru 0x8c3f26 (0x9927 bytes) 2024-09-15T00:09:47.102 INFO:teuthology.orchestra.run.smithi092.stdout:2662 trunc from 0x291bc00 to 0x577d400 2024-09-15T00:09:47.126 INFO:teuthology.orchestra.run.smithi092.stdout:2663 write 0x8339200 thru 0x83489d4 (0xf7d5 bytes) 2024-09-15T00:09:47.129 INFO:teuthology.orchestra.run.smithi092.stdout:2664 punch from 0x1447200 to 0x1449200, (0x2000 bytes) 2024-09-15T00:09:47.129 INFO:teuthology.orchestra.run.smithi092.stdout:2666 punch from 0x269b400 to 0x26a5400, (0xa000 bytes) 2024-09-15T00:09:47.131 INFO:teuthology.orchestra.run.smithi092.stdout:2668 punch from 0x5882800 to 0x5891a00, (0xf200 bytes) 2024-09-15T00:09:47.131 INFO:teuthology.orchestra.run.smithi092.stdout:2669 read 0x5f96000 thru 0x5f9cfdc (0x6fdd bytes) 2024-09-15T00:09:47.145 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:47.143+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:47.148 INFO:teuthology.orchestra.run.smithi092.stdout:2670 write 0xa8d5e00 thru 0xa8df269 (0x946a bytes) 2024-09-15T00:09:47.149 INFO:teuthology.orchestra.run.smithi092.stdout:2671 read 0xa782e00 thru 0xa78328f (0x490 bytes) 2024-09-15T00:09:47.153 INFO:teuthology.orchestra.run.smithi092.stdout:2675 trunc from 0xa8df400 to 0xd496400 2024-09-15T00:09:47.170 INFO:teuthology.orchestra.run.smithi092.stdout:2678 trunc from 0xd496400 to 0x8f80200 2024-09-15T00:09:47.175 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4070 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821983311, 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02702953, 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4070 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350999+0000", 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:47.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351003+0000", 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351007+0000", 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6820000000000001e-06 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351010+0000", 2024-09-15T00:09:47.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7269999999999999e-06 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377929+0000", 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026919887999999999 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377969+0000", 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9909e-05 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378028+0000", 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8578000000000001e-05 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4071 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821968430999998, 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027136008, 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.180 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4071 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351014+0000", 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351015+0000", 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7039999999999999e-06 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351020+0000", 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351023+0000", 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378053+0000", 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027029610999999999 2024-09-15T00:09:47.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378094+0000", 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1826999999999998e-05 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378149+0000", 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4815000000000003e-05 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4072 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821943751999999, 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027223404999999999, 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4072 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.185 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351040+0000", 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351046+0000", 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351051+0000", 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378171+0000", 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027120590999999999 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378209+0000", 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7308999999999998e-05 2024-09-15T00:09:47.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378261+0000", 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2753999999999997e-05 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4074 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821924856999999, 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027334509999999999, 2024-09-15T00:09:47.189 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4074 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9200000000000002e-07 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351059+0000", 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.012e-06 2024-09-15T00:09:47.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351065+0000", 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2129999999999996e-06 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351070+0000", 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8470000000000003e-06 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378285+0000", 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027214449000000002 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378334+0000", 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378391+0000", 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7046000000000002e-05 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4075 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821905986000001, 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027546923000000001, 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.194 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4075 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351078+0000", 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.905e-06 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351085+0000", 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351090+0000", 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378423+0000", 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027333051000000001 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378523+0000", 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9959999999999998e-05 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378623+0000", 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9962000000000003e-05 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4077 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821883302, 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027660619000000001, 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4077 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351099+0000", 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351101+0000", 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351107+0000", 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7980000000000002e-06 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351111+0000", 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378657+0000", 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027545463999999999 2024-09-15T00:09:47.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378702+0000", 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5809999999999997e-05 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378759+0000", 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4078 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821855301999999, 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027808405000000001, 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4078 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351127+0000", 2024-09-15T00:09:47.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351129+0000", 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351135+0000", 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3820000000000001e-06 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351141+0000", 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2259999999999996e-06 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378785+0000", 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027644592999999999 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378855+0000", 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0276000000000003e-05 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378935+0000", 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9361000000000001e-05 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4079 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821833665, 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027913778, 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4079 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351150+0000", 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9010000000000001e-06 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351157+0000", 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351162+0000", 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378958+0000", 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027795786999999999 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379004+0000", 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6124000000000001e-05 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379062+0000", 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7630999999999998e-05 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4081 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.211 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821809984000001, 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016658, 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4081 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351174+0000", 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351181+0000", 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9070000000000001e-06 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351186+0000", 2024-09-15T00:09:47.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.651e-06 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379085+0000", 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027898757999999999 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379130+0000", 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5126000000000001e-05 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379188+0000", 2024-09-15T00:09:47.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8548999999999998e-05 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4082 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821789930000001, 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028118457999999999, 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4082 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351194+0000", 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7519999999999999e-06 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351200+0000", 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.801e-06 2024-09-15T00:09:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351205+0000", 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379213+0000", 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028007539000000001 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379254+0000", 2024-09-15T00:09:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1817999999999998e-05 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379310+0000", 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5850000000000002e-05 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4083 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821768209999998, 2024-09-15T00:09:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028218838999999999, 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4083 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351214+0000", 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351216+0000", 2024-09-15T00:09:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351222+0000", 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351226+0000", 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379333+0000", 2024-09-15T00:09:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028106740000000002 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379376+0000", 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3581000000000002e-05 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379432+0000", 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5868000000000001e-05 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4085 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821739705999999, 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311823, 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4085 2024-09-15T00:09:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0699999999999998e-07 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351244+0000", 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.782e-06 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351248+0000", 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6289999999999999e-06 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351250+0000", 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379457+0000", 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028206479 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379499+0000", 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1637000000000001e-05 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379554+0000", 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5248999999999999e-05 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4086 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821727865, 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02842368, 2024-09-15T00:09:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4086 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351259+0000", 2024-09-15T00:09:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9520000000000002e-06 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351262+0000", 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351265+0000", 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379576+0000", 2024-09-15T00:09:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311228000000001 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379619+0000", 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2595000000000001e-05 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379678+0000", 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4087 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821713509999999, 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523903999999999, 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4087 2024-09-15T00:09:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351269+0000", 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351270+0000", 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1090000000000001e-06 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351273+0000", 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9189999999999999e-06 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351275+0000", 2024-09-15T00:09:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379701+0000", 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028425813000000001 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379742+0000", 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0769000000000002e-05 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379792+0000", 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4088 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821697375999999, 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028621284, 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4088 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351285+0000", 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351286+0000", 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.186e-06 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351289+0000", 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351292+0000", 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379815+0000", 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523211 2024-09-15T00:09:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379857+0000", 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1807999999999997e-05 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379906+0000", 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8594999999999998e-05 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4090 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821686290999999, 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028748587999999999, 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4090 2024-09-15T00:09:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351296+0000", 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351297+0000", 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1060000000000001e-06 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351300+0000", 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9110000000000002e-06 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351302+0000", 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379947+0000", 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028645177000000001 2024-09-15T00:09:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379991+0000", 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3680999999999997e-05 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380044+0000", 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2957000000000001e-05 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4091 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821676736000001, 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028864198000000001, 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4091 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351306+0000", 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351309+0000", 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351312+0000", 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380069+0000", 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028757933999999999 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380113+0000", 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3810000000000002e-05 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380169+0000", 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5887999999999998e-05 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4092 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821667378000001, 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028973819000000001, 2024-09-15T00:09:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4092 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351315+0000", 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9999999999999999e-07 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351316+0000", 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.95e-07 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351318+0000", 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8629999999999999e-06 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351321+0000", 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380195+0000", 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028873868 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380238+0000", 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2883999999999998e-05 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380288+0000", 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0096000000000003e-05 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4094 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821657759000001, 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02907964, 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4094 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351325+0000", 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.054e-06 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351328+0000", 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351331+0000", 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4679999999999999e-06 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380312+0000", 2024-09-15T00:09:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028981587999999999 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380355+0000", 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2688000000000002e-05 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380404+0000", 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8767999999999998e-05 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4095 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.821648371999999, 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029185068000000002, 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4095 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351334+0000", 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.04e-07 2024-09-15T00:09:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351335+0000", 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0300000000000001e-06 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351339+0000", 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351344+0000", 2024-09-15T00:09:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0560000000000004e-06 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380428+0000", 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02908444 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380470+0000", 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1338000000000003e-05 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380518+0000", 2024-09-15T00:09:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8724999999999999e-05 2024-09-15T00:09:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.265 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:47.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:47.267 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:47.263+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2679 trunc from 0x8f80200 to 0xd326c00 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2681 read 0x20de000 thru 0x20e1b31 (0x3b32 bytes) 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2682 read 0xbeb8c00 thru 0xbec5ef4 (0xd2f5 bytes) 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2683 trunc from 0xd326c00 to 0x2b4b800 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2688 read 0x12ae800 thru 0x12bda6d (0xf26e bytes) 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2689 write 0x176e000 thru 0x177d64b (0xf64c bytes) 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2692 read 0x1cbb400 thru 0x1cbfc02 (0x4803 bytes) 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2693 read 0x2267200 thru 0x2271eb3 (0xacb4 bytes) 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2695 write 0x8235c00 thru 0x823b7da (0x5bdb bytes) 2024-09-15T00:09:47.268 INFO:teuthology.orchestra.run.smithi092.stdout:2697 read 0x47e3000 thru 0x47ee87c (0xb87d bytes) 2024-09-15T00:09:47.269 INFO:teuthology.orchestra.run.smithi092.stdout:2699 write 0x3819400 thru 0x382372f (0xa330 bytes) 2024-09-15T00:09:47.272 INFO:teuthology.orchestra.run.smithi092.stdout:2700 write 0xaa9ca00 thru 0xaa9e8e3 (0x1ee4 bytes) 2024-09-15T00:09:47.273 INFO:teuthology.orchestra.run.smithi092.stdout:2704 write 0x86b0000 thru 0x86b7a5d (0x7a5e bytes) 2024-09-15T00:09:47.286 INFO:teuthology.orchestra.run.smithi092.stdout:2705 write 0xc11d600 thru 0xc127118 (0x9b19 bytes) 2024-09-15T00:09:47.287 INFO:teuthology.orchestra.run.smithi092.stdout:2709 read 0x625400 thru 0x62e985 (0x9586 bytes) 2024-09-15T00:09:47.287 INFO:teuthology.orchestra.run.smithi092.stdout:2710 write 0x12f3200 thru 0x12fc99c (0x979d bytes) 2024-09-15T00:09:47.295 INFO:teuthology.orchestra.run.smithi092.stdout:2711 trunc from 0xc127200 to 0x57d1600 2024-09-15T00:09:47.316 INFO:teuthology.orchestra.run.smithi092.stdout:2713 read 0x31de800 thru 0x31e866e (0x9e6f bytes) 2024-09-15T00:09:47.316 INFO:teuthology.orchestra.run.smithi092.stdout:2714 read 0x4c36000 thru 0x4c43724 (0xd725 bytes) 2024-09-15T00:09:47.326 INFO:teuthology.orchestra.run.smithi092.stdout:2716 write 0x63e9a00 thru 0x63eb059 (0x165a bytes) 2024-09-15T00:09:47.330 INFO:teuthology.orchestra.run.smithi092.stdout:2717 trunc from 0x63eb200 to 0x3c77000 2024-09-15T00:09:47.358 INFO:teuthology.orchestra.run.smithi092.stdout:2718 write 0x7350e00 thru 0x735d28a (0xc48b bytes) 2024-09-15T00:09:47.359 INFO:teuthology.orchestra.run.smithi092.stdout:2719 punch from 0x66ba600 to 0x66c8a00, (0xe400 bytes) 2024-09-15T00:09:47.361 INFO:teuthology.orchestra.run.smithi092.stdout:2721 write 0x488d600 thru 0x4897989 (0xa38a bytes) 2024-09-15T00:09:47.361 INFO:teuthology.orchestra.run.smithi092.stdout:2722 read 0x4d33800 thru 0x4d367f7 (0x2ff8 bytes) 2024-09-15T00:09:47.367 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:47.365+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:47.383 INFO:teuthology.orchestra.run.smithi092.stdout:2723 write 0x9f86400 thru 0x9f94870 (0xe471 bytes) 2024-09-15T00:09:47.384 INFO:teuthology.orchestra.run.smithi092.stdout:2724 read 0x2a77c00 thru 0x2a7b393 (0x3794 bytes) 2024-09-15T00:09:47.384 INFO:teuthology.orchestra.run.smithi092.stdout:2726 read 0x7555e00 thru 0x755616d (0x36e bytes) 2024-09-15T00:09:47.384 INFO:teuthology.orchestra.run.smithi092.stdout:2727 read 0x543b800 thru 0x544acc3 (0xf4c4 bytes) 2024-09-15T00:09:47.384 INFO:teuthology.orchestra.run.smithi092.stdout:2731 punch from 0x2287600 to 0x2287a00, (0x400 bytes) 2024-09-15T00:09:47.394 INFO:teuthology.orchestra.run.smithi092.stdout:2734 trunc from 0x9f94a00 to 0x2464c00 2024-09-15T00:09:47.418 INFO:teuthology.orchestra.run.smithi092.stdout:2736 trunc from 0x2464c00 to 0x4d03e00 2024-09-15T00:09:47.426 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4919.0:2427 3.9 3:91a8e8f2:::rbd_header.1129eb1f5358:head [notify-ack in=30b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:47.425402+0000", 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00040232499999999998, 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000416315, 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4919", 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3470824944", 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2427 2024-09-15T00:09:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425402+0000", 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425402+0000", 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425403+0000", 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1800000000000005e-07 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425406+0000", 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7129999999999999e-06 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425416+0000", 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7899999999999994e-06 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425422+0000", 2024-09-15T00:09:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2689999999999996e-06 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425671+0000", 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000248144 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425722+0000", 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1786000000000002e-05 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425818+0000", 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5795e-05 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:10067 3.9 3:91a8e8f2:::rbd_header.1129eb1f5358:head [notify-ack in=30b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:47.425433+0000", 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00037139699999999998, 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000443614, 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 10067 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425433+0000", 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425433+0000", 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425434+0000", 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425439+0000", 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9189999999999998e-06 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425451+0000", 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.202e-05 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425461+0000", 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.842e-06 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425837+0000", 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000376146 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425877+0000", 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0102000000000003e-05 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4919.0:2428 3.9 3.4f171589 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:47.425780+0000", 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.4553e-05, 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013349999999999999, 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4919", 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3470824944", 2024-09-15T00:09:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2428 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425780+0000", 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425780+0000", 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425780+0000", 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2900000000000003e-07 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425784+0000", 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6270000000000002e-06 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425789+0000", 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6350000000000003e-06 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:47.425797+0000", 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1550000000000001e-06 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:09:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 3 2024-09-15T00:09:47.440 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:47.441 INFO:teuthology.orchestra.run.smithi092.stdout:2738 punch from 0x2393600 to 0x2398000, (0x4a00 bytes) 2024-09-15T00:09:47.441 INFO:teuthology.orchestra.run.smithi092.stdout:2739 punch from 0x1fc7000 to 0x1fd5c00, (0xec00 bytes) 2024-09-15T00:09:47.441 INFO:teuthology.orchestra.run.smithi092.stdout:2740 punch from 0x3cd3800 to 0x3cdc800, (0x9000 bytes) 2024-09-15T00:09:47.441 INFO:teuthology.orchestra.run.smithi092.stdout:2742 punch from 0x236ce00 to 0x2371800, (0x4a00 bytes) 2024-09-15T00:09:47.441 INFO:teuthology.orchestra.run.smithi092.stdout:2743 trunc from 0x4d03e00 to 0x45df400 2024-09-15T00:09:47.442 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:47.447 INFO:teuthology.orchestra.run.smithi092.stdout:2745 read 0x95800 thru 0x9d2e4 (0x7ae5 bytes) 2024-09-15T00:09:47.461 INFO:teuthology.orchestra.run.smithi092.stdout:2746 write 0x73dd400 thru 0x73ebfde (0xebdf bytes) 2024-09-15T00:09:47.468 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:47.466+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:2747 write 0xb8d4a00 thru 0xb8df363 (0xa964 bytes) 2024-09-15T00:09:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:2748 read 0xa895c00 thru 0xa89c8c1 (0x6cc2 bytes) 2024-09-15T00:09:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:2750 read 0x7e97800 thru 0x7ea6a55 (0xf256 bytes) 2024-09-15T00:09:47.488 INFO:teuthology.orchestra.run.smithi092.stdout:2751 trunc from 0xb8df400 to 0x4615a00 2024-09-15T00:09:47.517 INFO:teuthology.orchestra.run.smithi092.stdout:2753 trunc from 0x4615a00 to 0xd404a00 2024-09-15T00:09:47.526 INFO:teuthology.orchestra.run.smithi092.stdout:2754 punch from 0x2bd1c00 to 0x2bd6800, (0x4c00 bytes) 2024-09-15T00:09:47.531 INFO:teuthology.orchestra.run.smithi092.stdout:2756 write 0xad55400 thru 0xad6112c (0xbd2d bytes) 2024-09-15T00:09:47.531 INFO:teuthology.orchestra.run.smithi092.stdout:2757 punch from 0x3c14000 to 0x3c17a00, (0x3a00 bytes) 2024-09-15T00:09:47.531 INFO:teuthology.orchestra.run.smithi092.stdout:2759 read 0x96ef000 thru 0x96f23f3 (0x33f4 bytes) 2024-09-15T00:09:47.531 INFO:teuthology.orchestra.run.smithi092.stdout:2761 read 0x33c4400 thru 0x33d312b (0xed2c bytes) 2024-09-15T00:09:47.531 INFO:teuthology.orchestra.run.smithi092.stdout:2763 punch from 0xc86de00 to 0xc870600, (0x2800 bytes) 2024-09-15T00:09:47.532 INFO:teuthology.orchestra.run.smithi092.stdout:2767 read 0x8bcf600 thru 0x8bd9bb6 (0xa5b7 bytes) 2024-09-15T00:09:47.535 INFO:teuthology.orchestra.run.smithi092.stdout:2768 trunc from 0xd404a00 to 0xa118800 2024-09-15T00:09:47.554 INFO:teuthology.orchestra.run.smithi092.stdout:2769 read 0x535fe00 thru 0x5361469 (0x166a bytes) 2024-09-15T00:09:47.556 INFO:teuthology.orchestra.run.smithi092.stdout:2770 read 0x40d2600 thru 0x40dcdfd (0xa7fe bytes) 2024-09-15T00:09:47.556 INFO:teuthology.orchestra.run.smithi092.stdout:2771 write 0x959be00 thru 0x95a5973 (0x9b74 bytes) 2024-09-15T00:09:47.556 INFO:teuthology.orchestra.run.smithi092.stdout:2773 read 0x5ac0a00 thru 0x5acded7 (0xd4d8 bytes) 2024-09-15T00:09:47.562 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:47.562 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:47.562 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:47.566 INFO:teuthology.orchestra.run.smithi092.stdout:2774 trunc from 0xa118800 to 0xe69800 2024-09-15T00:09:47.569 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:47.566+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:47.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:47.601 INFO:teuthology.orchestra.run.smithi092.stdout:2776 trunc from 0xe69800 to 0x4191800 2024-09-15T00:09:47.611 INFO:teuthology.orchestra.run.smithi092.stdout:2778 punch from 0xd2f800 to 0xd3c200, (0xca00 bytes) 2024-09-15T00:09:47.622 INFO:teuthology.orchestra.run.smithi092.stdout:2779 trunc from 0x4191800 to 0x73a0800 2024-09-15T00:09:47.631 INFO:teuthology.orchestra.run.smithi092.stdout:2782 read 0x5d89800 thru 0x5d96bda (0xd3db bytes) 2024-09-15T00:09:47.638 INFO:teuthology.orchestra.run.smithi092.stdout:2783 write 0x172a400 thru 0x1736e19 (0xca1a bytes) 2024-09-15T00:09:47.656 INFO:teuthology.orchestra.run.smithi092.stdout:2785 write 0xbe7e000 thru 0xbe840ee (0x60ef bytes) 2024-09-15T00:09:47.657 INFO:teuthology.orchestra.run.smithi092.stdout:2787 read 0x6111400 thru 0x6119314 (0x7f15 bytes) 2024-09-15T00:09:47.657 INFO:teuthology.orchestra.run.smithi092.stdout:2788 read 0x18c4200 thru 0x18c5d58 (0x1b59 bytes) 2024-09-15T00:09:47.657 INFO:teuthology.orchestra.run.smithi092.stdout:2789 read 0x58c9200 thru 0x58ccf7a (0x3d7b bytes) 2024-09-15T00:09:47.657 INFO:teuthology.orchestra.run.smithi092.stdout:2792 read 0x866f000 thru 0x867d82e (0xe82f bytes) 2024-09-15T00:09:47.657 INFO:teuthology.orchestra.run.smithi092.stdout:2793 punch from 0x3ba7000 to 0x3bb4e00, (0xde00 bytes) 2024-09-15T00:09:47.658 INFO:teuthology.orchestra.run.smithi092.stdout:2795 read 0x4a64e00 thru 0x4a66945 (0x1b46 bytes) 2024-09-15T00:09:47.658 INFO:teuthology.orchestra.run.smithi092.stdout:2796 read 0x386b200 thru 0x386d46d (0x226e bytes) 2024-09-15T00:09:47.658 INFO:teuthology.orchestra.run.smithi092.stdout:2797 read 0x2f9b000 thru 0x2fa5723 (0xa724 bytes) 2024-09-15T00:09:47.658 INFO:teuthology.orchestra.run.smithi092.stdout:2799 write 0x4c14800 thru 0x4c23ca9 (0xf4aa bytes) 2024-09-15T00:09:47.658 INFO:teuthology.orchestra.run.smithi092.stdout:2801 punch from 0x8247200 to 0x8255800, (0xe600 bytes) 2024-09-15T00:09:47.659 INFO:teuthology.orchestra.run.smithi092.stdout:skipping zero length punch hole 2024-09-15T00:09:47.659 INFO:teuthology.orchestra.run.smithi092.stdout:2808 write 0x736fc00 thru 0x73747c6 (0x4bc7 bytes) 2024-09-15T00:09:47.667 INFO:teuthology.orchestra.run.smithi092.stdout:2809 trunc from 0xbe84200 to 0x4d14a00 2024-09-15T00:09:47.669 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:47.667+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:47.688 INFO:teuthology.orchestra.run.smithi092.stdout:2811 read 0x3a4d400 thru 0x3a5a637 (0xd238 bytes) 2024-09-15T00:09:47.688 INFO:teuthology.orchestra.run.smithi092.stdout:2814 punch from 0x126800 to 0x12d800, (0x7000 bytes) 2024-09-15T00:09:47.690 INFO:teuthology.orchestra.run.smithi092.stdout:2816 read 0x12f9800 thru 0x13042a6 (0xaaa7 bytes) 2024-09-15T00:09:47.690 INFO:teuthology.orchestra.run.smithi092.stdout:2817 write 0x2858c00 thru 0x2861966 (0x8d67 bytes) 2024-09-15T00:09:47.690 INFO:teuthology.orchestra.run.smithi092.stdout:2819 write 0x435ae00 thru 0x4361408 (0x6609 bytes) 2024-09-15T00:09:47.690 INFO:teuthology.orchestra.run.smithi092.stdout:2820 punch from 0xdee000 to 0xdf9e00, (0xbe00 bytes) 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3427 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403392205999999, 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027724631, 2024-09-15T00:09:47.694 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3427 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287582+0000", 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2599999999999998e-07 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287584+0000", 2024-09-15T00:09:47.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8759999999999999e-06 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287589+0000", 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287594+0000", 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.357e-06 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315217+0000", 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027622785 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315257+0000", 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0228000000000002e-05 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315306+0000", 2024-09-15T00:09:47.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9228999999999999e-05 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3428 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403374200999998, 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027928562000000001, 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.699 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3428 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:47.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287600+0000", 2024-09-15T00:09:47.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:47.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287602+0000", 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287608+0000", 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287613+0000", 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8740000000000003e-06 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315346+0000", 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027733099000000001 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315431+0000", 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4350000000000004e-05 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315528+0000", 2024-09-15T00:09:47.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7374000000000007e-05 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3429 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403354890999999, 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028028630999999998, 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.704 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3429 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287621+0000", 2024-09-15T00:09:47.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287627+0000", 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8440000000000003e-06 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287631+0000", 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5009999999999998e-06 2024-09-15T00:09:47.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315556+0000", 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027925241999999999 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315595+0000", 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8512e-05 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315647+0000", 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2318000000000003e-05 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3430 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403336817, 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028122114, 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3430 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287644+0000", 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4219999999999997e-06 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287650+0000", 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0920000000000003e-06 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287654+0000", 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4279999999999998e-06 2024-09-15T00:09:47.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315671+0000", 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016592 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315709+0000", 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8231e-05 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315759+0000", 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9926999999999998e-05 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3431 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403313276999999, 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02831842, 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3431 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287661+0000", 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287663+0000", 2024-09-15T00:09:47.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9570000000000001e-06 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287668+0000", 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287673+0000", 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315799+0000", 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028125741999999999 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315891+0000", 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2701000000000006e-05 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315979+0000", 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7287000000000006e-05 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3432 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403294893000002, 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028430624000000002, 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3432 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287681+0000", 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8530000000000001e-06 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287686+0000", 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4659999999999998e-06 2024-09-15T00:09:47.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287691+0000", 2024-09-15T00:09:47.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2230000000000001e-06 2024-09-15T00:09:47.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316006+0000", 2024-09-15T00:09:47.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028315344999999999 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316054+0000", 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8004999999999999e-05 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316109+0000", 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5374999999999998e-05 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3433 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403277137, 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028528593000000001, 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3433 2024-09-15T00:09:47.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287697+0000", 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287699+0000", 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8509999999999999e-06 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287704+0000", 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287710+0000", 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316124+0000", 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028414466999999999 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316171+0000", 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6783999999999998e-05 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316225+0000", 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3891999999999998e-05 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3434 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403257752000002, 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028632801999999999, 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3434 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-07 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287718+0000", 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:47.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287724+0000", 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8390000000000003e-06 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287729+0000", 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8389999999999997e-06 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316250+0000", 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028521135 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316295+0000", 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4540999999999999e-05 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316349+0000", 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4083999999999999e-05 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3435 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403227076, 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028723243999999998, 2024-09-15T00:09:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3435 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287747+0000", 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4900000000000002e-07 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287748+0000", 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4190000000000001e-06 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287752+0000", 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7900000000000001e-06 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287756+0000", 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1609999999999996e-06 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316375+0000", 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028618932 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316417+0000", 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316470+0000", 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.737 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3436 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403212027999999, 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028827854, 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3436 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287766+0000", 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287771+0000", 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5140000000000001e-06 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287776+0000", 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316494+0000", 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028718144000000001 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316538+0000", 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4129999999999999e-05 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316589+0000", 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1548999999999997e-05 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3437 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403194136, 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028972188999999999, 2024-09-15T00:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3437 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287780+0000", 2024-09-15T00:09:47.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287782+0000", 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8419999999999999e-06 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287787+0000", 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3059999999999997e-06 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287791+0000", 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9920000000000001e-06 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316615+0000", 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028824487999999999 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316668+0000", 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2883999999999997e-05 2024-09-15T00:09:47.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316752+0000", 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3308999999999998e-05 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3438 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403177094, 2024-09-15T00:09:47.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029124331999999999, 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3438 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:47.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287797+0000", 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287799+0000", 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287804+0000", 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7069999999999997e-06 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287808+0000", 2024-09-15T00:09:47.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0720000000000001e-06 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316799+0000", 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028990241999999999 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316845+0000", 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5837000000000002e-05 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316921+0000", 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6255999999999996e-05 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3439 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403159554999998, 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029290665, 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3439 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287816+0000", 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6980000000000001e-06 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287824+0000", 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2500000000000006e-06 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287829+0000", 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:09:47.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316967+0000", 2024-09-15T00:09:47.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029138319999999999 2024-09-15T00:09:47.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317047+0000", 2024-09-15T00:09:47.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0081999999999997e-05 2024-09-15T00:09:47.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317105+0000", 2024-09-15T00:09:47.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7497999999999997e-05 2024-09-15T00:09:47.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.758 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3440 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.758 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:47.758 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403138496, 2024-09-15T00:09:47.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029395451999999999, 2024-09-15T00:09:47.758 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.758 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3440 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287837+0000", 2024-09-15T00:09:47.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287843+0000", 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6799999999999998e-06 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287848+0000", 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317131+0000", 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029283337999999999 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317176+0000", 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5176999999999998e-05 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317231+0000", 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4120999999999999e-05 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3441 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403120432000001, 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029496321999999998, 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.763 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3441 2024-09-15T00:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:47.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:47.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:09:47.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287855+0000", 2024-09-15T00:09:47.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7099999999999999e-06 2024-09-15T00:09:47.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287861+0000", 2024-09-15T00:09:47.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287866+0000", 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317252+0000", 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029386051999999999 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317297+0000", 2024-09-15T00:09:47.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5164999999999999e-05 2024-09-15T00:09:47.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317349+0000", 2024-09-15T00:09:47.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2534000000000002e-05 2024-09-15T00:09:47.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3442 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.769 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:47.769 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403101165999999, 2024-09-15T00:09:47.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029594697999999999, 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3442 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287873+0000", 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287879+0000", 2024-09-15T00:09:47.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287885+0000", 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.755e-06 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287890+0000", 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317375+0000", 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029485457 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317416+0000", 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1217999999999997e-05 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317467+0000", 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0819999999999998e-05 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3443 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403077827000001, 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029684676, 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.774 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3443 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287898+0000", 2024-09-15T00:09:47.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287904+0000", 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8860000000000001e-06 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287908+0000", 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5650000000000003e-06 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317488+0000", 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029579437 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317530+0000", 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1900999999999997e-05 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317580+0000", 2024-09-15T00:09:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0603000000000003e-05 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3444 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403052609, 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029772823, 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3444 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:47.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287931+0000", 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0008e-05 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287938+0000", 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287943+0000", 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317603+0000", 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029659913999999999 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317646+0000", 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2367999999999998e-05 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317694+0000", 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8201999999999997e-05 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3445 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403023270999999, 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029863565000000002, 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3445 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287951+0000", 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:47.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287953+0000", 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.765e-06 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287958+0000", 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999996e-06 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287963+0000", 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.634e-06 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317714+0000", 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029751353000000001 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317755+0000", 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0342000000000001e-05 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317814+0000", 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9188999999999999e-05 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3446 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.403004973000002, 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029967884, 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:47.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3446 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287971+0000", 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287977+0000", 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4330000000000003e-06 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287981+0000", 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3989999999999997e-06 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317837+0000", 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029855942 2024-09-15T00:09:47.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317878+0000", 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467000000000001e-05 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317936+0000", 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8100999999999998e-05 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:47.792 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:47.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:47.795 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:47.792+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:47.795 INFO:teuthology.orchestra.run.smithi092.stdout:2821 trunc from 0x4d14a00 to 0xa401a00 2024-09-15T00:09:47.795 INFO:teuthology.orchestra.run.smithi092.stdout:2822 trunc from 0xa401a00 to 0x696000 2024-09-15T00:09:47.795 INFO:teuthology.orchestra.run.smithi092.stdout:2823 write 0x9058800 thru 0x9066c73 (0xe474 bytes) 2024-09-15T00:09:47.795 INFO:teuthology.orchestra.run.smithi092.stdout:2824 write 0xa3b4600 thru 0xa3c230b (0xdd0c bytes) 2024-09-15T00:09:47.796 INFO:teuthology.orchestra.run.smithi092.stdout:2825 read 0x9e23200 thru 0x9e3186b (0xe66c bytes) 2024-09-15T00:09:47.796 INFO:teuthology.orchestra.run.smithi092.stdout:2826 punch from 0x2eafc00 to 0x2eb5800, (0x5c00 bytes) 2024-09-15T00:09:47.796 INFO:teuthology.orchestra.run.smithi092.stdout:2828 read 0xa2cca00 thru 0xa2d6a58 (0xa059 bytes) 2024-09-15T00:09:47.803 INFO:teuthology.orchestra.run.smithi092.stdout:2829 write 0xd7bd200 thru 0xd7c3355 (0x6156 bytes) 2024-09-15T00:09:47.807 INFO:teuthology.orchestra.run.smithi092.stdout:2831 trunc from 0xd7c3400 to 0xac96e00 2024-09-15T00:09:47.825 INFO:teuthology.orchestra.run.smithi092.stdout:2835 write 0x71e9400 thru 0x71f317d (0x9d7e bytes) 2024-09-15T00:09:47.830 INFO:teuthology.orchestra.run.smithi092.stdout:2837 clone 22 order 25 su 8388608 sc 3 2024-09-15T00:09:47.895 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:47.893+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:47.938 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:47.938 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:47.946 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:47.982 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:09:48.031 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:48.030+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:48.046 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone21 from 0xac96e00 (overlap 0x156200) to 0x66a00 2024-09-15T00:09:48.097 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:48.096+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:48.102 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:48.103 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:48.103 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:48.103 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:48.103 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:48.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:48.198 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:48.196+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:48.259 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1142 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.374753179999999, 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045190363999999997, 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.261 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1142 2024-09-15T00:09:48.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:48.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:48.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882756+0000", 2024-09-15T00:09:48.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-15T00:09:48.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882773+0000", 2024-09-15T00:09:48.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6872999999999999e-05 2024-09-15T00:09:48.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882780+0000", 2024-09-15T00:09:48.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4240000000000004e-06 2024-09-15T00:09:48.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882787+0000", 2024-09-15T00:09:48.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8510000000000001e-06 2024-09-15T00:09:48.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927799+0000", 2024-09-15T00:09:48.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045011974000000003 2024-09-15T00:09:48.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927843+0000", 2024-09-15T00:09:48.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4270999999999997e-05 2024-09-15T00:09:48.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927946+0000", 2024-09-15T00:09:48.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102581 2024-09-15T00:09:48.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.270 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.271 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.271 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:48.271 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.374704487999999, 2024-09-15T00:09:48.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:09:48.272 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.272 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.272 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.273 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:09:48.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.273 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:48.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:48.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:09:48.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:09:48.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:09:48.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:09:48.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:09:48.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:09:48.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:09:48.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:09:48.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:09:48.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:09:48.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:09:48.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:09:48.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:09:48.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:09:48.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.282 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.282 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:48.283 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.374665858, 2024-09-15T00:09:48.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:09:48.283 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.283 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.283 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.284 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:09:48.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.284 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:48.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:48.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:48.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:09:48.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:09:48.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:09:48.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:09:48.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:09:48.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:09:48.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:09:48.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:09:48.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:09:48.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:09:48.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:09:48.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:09:48.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:09:48.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.293 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.293 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.293 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:48.294 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.374639921, 2024-09-15T00:09:48.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:09:48.294 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.294 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.294 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.295 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.295 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.295 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:09:48.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.295 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:48.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:48.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:48.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:09:48.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:09:48.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:09:48.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:09:48.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:09:48.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:09:48.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:09:48.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:09:48.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:09:48.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:09:48.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:09:48.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:09:48.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:09:48.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.304 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.304 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:48.305 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.374623852999999, 2024-09-15T00:09:48.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:09:48.305 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.306 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:09:48.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.306 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:48.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:48.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:09:48.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:09:48.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:09:48.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:48.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:09:48.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:09:48.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:09:48.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:09:48.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:09:48.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:09:48.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:09:48.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:09:48.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:09:48.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:09:48.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.314 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.315 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.315 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:48.315 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.374482757999999, 2024-09-15T00:09:48.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:09:48.316 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.316 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.316 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.317 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.317 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.317 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:09:48.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.317 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:48.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:48.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:09:48.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:09:48.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:09:48.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:48.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:09:48.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:09:48.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:09:48.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:09:48.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:09:48.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.373711003, 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:09:48.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:48.328 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.373625971999999, 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:09:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.373590956000001, 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:09:48.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:48.337 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.373565353, 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:48.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:09:48.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.373391087000002, 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:09:48.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:09:48.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.373159948000001, 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:09:48.346 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:09:48.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:09:48.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1157 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.372378510000001, 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045184281, 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1157 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.351 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885132+0000", 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885139+0000", 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0350000000000004e-06 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885153+0000", 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3515e-05 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930177+0000", 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045024495999999997 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930249+0000", 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2072999999999999e-05 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930314+0000", 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4826999999999994e-05 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.372348003999999, 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.355 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:09:48.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:09:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.372268263999999, 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:09:48.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.372063238999999, 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.364 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:09:48.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:09:48.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.371851706000001, 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:09:48.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.373 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.369295698999998, 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:09:48.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.367312593000001, 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:48.382 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.367293021999998, 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:48.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:48.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:48.387 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:48.387 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:48.387 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-15T00:09:48.388 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-15T00:09:48.388 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:48.387+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:48.389 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:48.489 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:48.487+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:48.519 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5137.0:2557 3.19 3.2367ef9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:48.518652+0000", 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.6308999999999999e-05, 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3318000000000001e-05, 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5137", 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1414811038", 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2557 2024-09-15T00:09:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:48.518652+0000", 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:48.518652+0000", 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:48.518652+0000", 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5699999999999998e-07 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:48.518655+0000", 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5450000000000002e-06 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:48.518661+0000", 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1260000000000003e-06 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:48.518665+0000", 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.659e-06 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:48.518700+0000", 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5037999999999999e-05 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 1 2024-09-15T00:09:48.523 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:48.529 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:48.614 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:48.612+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:48.690 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:48.688+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:48.697 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:48.697 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:48.697 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:48.697 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:48.697 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:48.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:48.748 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:48.790 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:48.789+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:09:48.869 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:48.869 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:48.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:48.869 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:48.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.869 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839232215999999, 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:09:48.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839211961, 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:09:48.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839192215000001, 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:09:48.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.883 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839172264999998, 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:09:48.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839144765, 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.892 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839127924, 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:09:48.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839111926000001, 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:09:48.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839088938, 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.902 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:09:48.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839063054, 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:09:48.906 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:09:48.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.910 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839046626999998, 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:09:48.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:48.915 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839032356000001, 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:48.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.919 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.839023039000001, 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.920 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:09:48.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:48.924 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.837354027, 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:48.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:09:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.928 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.837318964000001, 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:09:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.837293936999998, 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:09:48.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.837269629000001, 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.938 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:09:48.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2674 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3768320~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.942 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.835123283000001, 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059850591000000002, 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2674 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031671+0000", 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031682+0000", 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1248e-05 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031693+0000", 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1395e-05 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031705+0000", 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2106000000000001e-05 2024-09-15T00:09:48.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091352+0000", 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059646421999999998 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091441+0000", 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9174999999999999e-05 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091521+0000", 2024-09-15T00:09:48.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9636999999999997e-05 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.835077378000001, 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.947 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:48.951 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.835053246000001, 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:09:48.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.835023874000001, 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:48.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:09:48.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:48.960 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:48.962 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:48.960+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:48.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:48.963 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:48.963 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:48.963 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:48.964 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:48.964 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:48.964 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:48.964 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:48.965 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:48.965 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:48.965 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:09:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:48.966 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:09:49.037 INFO:teuthology.orchestra.run.smithi092.stdout:2839 write 0xe90a00 thru 0xe98699 (0x7c9a bytes) 2024-09-15T00:09:49.037 INFO:teuthology.orchestra.run.smithi092.stdout:2840 write 0x8ff2c00 thru 0x8ffc5bb (0x99bc bytes) 2024-09-15T00:09:49.038 INFO:teuthology.orchestra.run.smithi092.stdout:2841 write 0x426bc00 thru 0x42734d5 (0x78d6 bytes) 2024-09-15T00:09:49.046 INFO:teuthology.orchestra.run.smithi092.stdout:2843 trunc from 0xac96e00 to 0x3967600 2024-09-15T00:09:49.063 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:49.061+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:49.074 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T00:09:49.074 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-15T00:09:49.074 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:09:49.074 DEBUG:teuthology.orchestra.run.smithi027:> type ceph-objectstore-tool 2024-09-15T00:09:49.080 INFO:teuthology.orchestra.run.smithi092.stdout:2845 trunc from 0x3967600 to 0x6f53a00 2024-09-15T00:09:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T00:09:49.090 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-15T00:09:49.091 INFO:teuthology.orchestra.run.smithi092.stdout:2847 flatten 2024-09-15T00:09:49.109 INFO:teuthology.orchestra.run.smithi092.stdout:2850 trunc from 0x6f53a00 to 0x10a0600 2024-09-15T00:09:49.118 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:49.198 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:49.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:49.202 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:49.199+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:49.203 INFO:teuthology.orchestra.run.smithi092.stdout:2856 punch from 0x2ba800 to 0x2c0200, (0x5a00 bytes) 2024-09-15T00:09:49.203 INFO:teuthology.orchestra.run.smithi092.stdout:2859 trunc from 0x10a0600 to 0x1bcd400 2024-09-15T00:09:49.203 INFO:teuthology.orchestra.run.smithi092.stdout:2861 read 0x107e00 thru 0x113670 (0xb871 bytes) 2024-09-15T00:09:49.203 INFO:teuthology.orchestra.run.smithi092.stdout:2862 read 0x6cfc00 thru 0x6d941a (0x981b bytes) 2024-09-15T00:09:49.203 INFO:teuthology.orchestra.run.smithi092.stdout:2863 trunc from 0x1bcd400 to 0x1c79e00 2024-09-15T00:09:49.203 INFO:teuthology.orchestra.run.smithi092.stdout:2865 write 0x2b5c000 thru 0x2b6217a (0x617b bytes) 2024-09-15T00:09:49.204 INFO:teuthology.orchestra.run.smithi092.stdout:2866 read 0x1a99600 thru 0x1aa499a (0xb39b bytes) 2024-09-15T00:09:49.204 INFO:teuthology.orchestra.run.smithi092.stdout:2871 clone 23 order 23 su 1048576 sc 11 2024-09-15T00:09:49.300 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:49.298+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:49.348 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:49.348 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:49.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:49.401 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:49.399+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:49.502 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:49.500+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:49.503 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:49.503 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:49.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:49.603 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:49.602+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:49.617 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:49.617 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:49.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:49.704 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:49.702+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:49.766 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:49.767 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:49.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:49.818 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:49.817+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:49.877 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:49.877 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:49.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:49.905 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:49.903+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:49.996 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:09:50.037 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:50.037 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:50.041 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:50.039+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:50.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:50.107 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:50.105+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:50.153 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:50.153 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:50.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:50.209 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:50.207+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:50.268 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:50.268 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:50.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:50.310 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:50.309+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:50.334 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-15T00:09:50.409 INFO:teuthology.orchestra.run.smithi092.stdout:2875 trunc from 0x2b62200 to 0xe599000 2024-09-15T00:09:50.413 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:50.411+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:50.423 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:50.423 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:50.432 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:50.454 INFO:teuthology.orchestra.run.smithi092.stdout:2877 punch from 0x6e88000 to 0x6e88600, (0x600 bytes) 2024-09-15T00:09:50.457 INFO:teuthology.orchestra.run.smithi092.stdout:2878 write 0x7647600 thru 0x764f830 (0x8231 bytes) 2024-09-15T00:09:50.463 INFO:teuthology.orchestra.run.smithi092.stdout:2879 trunc from 0xe599000 to 0x93e1000 2024-09-15T00:09:50.475 INFO:teuthology.orchestra.run.smithi092.stdout:2883 read 0x1ea5e00 thru 0x1eaa460 (0x4661 bytes) 2024-09-15T00:09:50.480 INFO:teuthology.orchestra.run.smithi092.stdout:2886 trunc from 0x93e1000 to 0xaf2b400 2024-09-15T00:09:50.490 INFO:teuthology.orchestra.run.smithi092.stdout:2887 write 0xa844e00 thru 0xa849a46 (0x4c47 bytes) 2024-09-15T00:09:50.492 INFO:teuthology.orchestra.run.smithi092.stdout:2889 trunc from 0xaf2b400 to 0xc446a00 2024-09-15T00:09:50.505 INFO:teuthology.orchestra.run.smithi092.stdout:2893 read 0x8d8dc00 thru 0x8d92548 (0x4949 bytes) 2024-09-15T00:09:50.506 INFO:teuthology.orchestra.run.smithi092.stdout:2895 punch from 0x9bf2000 to 0x9bfb400, (0x9400 bytes) 2024-09-15T00:09:50.507 INFO:teuthology.orchestra.run.smithi092.stdout:2897 read 0xa268800 thru 0xa26d840 (0x5041 bytes) 2024-09-15T00:09:50.508 INFO:teuthology.orchestra.run.smithi092.stdout:2898 write 0x5ccf400 thru 0x5cd8950 (0x9551 bytes) 2024-09-15T00:09:50.508 INFO:teuthology.orchestra.run.smithi092.stdout:2900 read 0x517e00 thru 0x522e03 (0xb004 bytes) 2024-09-15T00:09:50.509 INFO:teuthology.orchestra.run.smithi092.stdout:2901 write 0x84d7000 thru 0x84d753b (0x53c bytes) 2024-09-15T00:09:50.509 INFO:teuthology.orchestra.run.smithi092.stdout:2905 punch from 0x57a0e00 to 0x57a6e00, (0x6000 bytes) 2024-09-15T00:09:50.510 INFO:teuthology.orchestra.run.smithi092.stdout:2906 punch from 0x8c5c400 to 0x8c6b400, (0xf000 bytes) 2024-09-15T00:09:50.513 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:50.511+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:50.517 INFO:teuthology.orchestra.run.smithi092.stdout:2910 trunc from 0xc446a00 to 0x6562600 2024-09-15T00:09:50.577 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:50.577 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:50.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:50.629 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:50.628+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:50.668 INFO:teuthology.orchestra.run.smithi092.stdout:2914 read 0x3eff600 thru 0x3f04e18 (0x5819 bytes) 2024-09-15T00:09:50.668 INFO:teuthology.orchestra.run.smithi092.stdout:2915 read 0x6332a00 thru 0x634109b (0xe69c bytes) 2024-09-15T00:09:50.688 INFO:teuthology.orchestra.run.smithi092.stdout:2916 write 0x92b1600 thru 0x92bc27d (0xac7e bytes) 2024-09-15T00:09:50.696 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:50.696 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:50.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:50.710 INFO:teuthology.orchestra.run.smithi092.stdout:2917 write 0xd978600 thru 0xd98801d (0xfa1e bytes) 2024-09-15T00:09:50.710 INFO:teuthology.orchestra.run.smithi092.stdout:2920 write 0x55e7c00 thru 0x55f6eae (0xf2af bytes) 2024-09-15T00:09:50.710 INFO:teuthology.orchestra.run.smithi092.stdout:2921 read 0x9e5ca00 thru 0x9e634cc (0x6acd bytes) 2024-09-15T00:09:50.711 INFO:teuthology.orchestra.run.smithi092.stdout:2923 write 0xb78ca00 thru 0xb790149 (0x374a bytes) 2024-09-15T00:09:50.711 INFO:teuthology.orchestra.run.smithi092.stdout:2924 read 0x249ea00 thru 0x24a3c49 (0x524a bytes) 2024-09-15T00:09:50.711 INFO:teuthology.orchestra.run.smithi092.stdout:2926 read 0xf82c00 thru 0xf90b0b (0xdf0c bytes) 2024-09-15T00:09:50.727 INFO:teuthology.orchestra.run.smithi092.stdout:2927 trunc from 0xd988200 to 0xc1be00 2024-09-15T00:09:50.748 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:50.746+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:50.786 INFO:teuthology.orchestra.run.smithi092.stdout:2930 write 0xc7ddc00 thru 0xc7e1c5d (0x405e bytes) 2024-09-15T00:09:50.799 INFO:teuthology.orchestra.run.smithi092.stdout:2932 write 0xdc3a400 thru 0xdc47ba6 (0xd7a7 bytes) 2024-09-15T00:09:50.799 INFO:teuthology.orchestra.run.smithi092.stdout:2934 read 0xb312200 thru 0xb321c3c (0xfa3d bytes) 2024-09-15T00:09:50.813 INFO:teuthology.orchestra.run.smithi092.stdout:2935 trunc from 0xdc47c00 to 0x27eb400 2024-09-15T00:09:50.815 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:50.813+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:50.821 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:50.821 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:50.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:50.832 INFO:teuthology.orchestra.run.smithi092.stdout:2939 punch from 0x201b400 to 0x2022e00, (0x7a00 bytes) 2024-09-15T00:09:50.843 INFO:teuthology.orchestra.run.smithi092.stdout:2941 write 0x44d1e00 thru 0x44dd047 (0xb248 bytes) 2024-09-15T00:09:50.849 INFO:teuthology.orchestra.run.smithi092.stdout:2944 punch from 0x1e20000 to 0x1e26800, (0x6800 bytes) 2024-09-15T00:09:50.849 INFO:teuthology.orchestra.run.smithi092.stdout:2945 read 0x1e6a00 thru 0x1f3a1b (0xd01c bytes) 2024-09-15T00:09:50.850 INFO:teuthology.orchestra.run.smithi092.stdout:2947 read 0x3f66200 thru 0x3f6b3f7 (0x51f8 bytes) 2024-09-15T00:09:50.851 INFO:teuthology.orchestra.run.smithi092.stdout:2949 write 0x87800 thru 0x88aed (0x12ee bytes) 2024-09-15T00:09:50.851 INFO:teuthology.orchestra.run.smithi092.stdout:2952 punch from 0x10ef200 to 0x10f3000, (0x3e00 bytes) 2024-09-15T00:09:50.851 INFO:teuthology.orchestra.run.smithi092.stdout:2953 read 0x388200 thru 0x393ea6 (0xbca7 bytes) 2024-09-15T00:09:50.852 INFO:teuthology.orchestra.run.smithi092.stdout:2955 punch from 0x38c0200 to 0x38c1000, (0xe00 bytes) 2024-09-15T00:09:50.852 INFO:teuthology.orchestra.run.smithi092.stdout:2956 read 0x10f4a00 thru 0x10f5149 (0x74a bytes) 2024-09-15T00:09:50.853 INFO:teuthology.orchestra.run.smithi092.stdout:2958 punch from 0xb33800 to 0xb3b600, (0x7e00 bytes) 2024-09-15T00:09:50.853 INFO:teuthology.orchestra.run.smithi092.stdout:2959 punch from 0xc2ae00 to 0xc2d000, (0x2200 bytes) 2024-09-15T00:09:50.853 INFO:teuthology.orchestra.run.smithi092.stdout:2960 punch from 0x878400 to 0x883000, (0xac00 bytes) 2024-09-15T00:09:50.854 INFO:teuthology.orchestra.run.smithi092.stdout:2961 punch from 0x11b0400 to 0x11be600, (0xe200 bytes) 2024-09-15T00:09:50.854 INFO:teuthology.orchestra.run.smithi092.stdout:2964 read 0x40a7200 thru 0x40b5710 (0xe511 bytes) 2024-09-15T00:09:50.863 INFO:teuthology.orchestra.run.smithi092.stdout:2967 trunc from 0x44dd200 to 0xc938a00 2024-09-15T00:09:50.871 INFO:teuthology.orchestra.run.smithi092.stdout:2968 read 0x189800 thru 0x190847 (0x7048 bytes) 2024-09-15T00:09:50.878 INFO:teuthology.orchestra.run.smithi092.stdout:2969 read 0x4ffc00 thru 0x5007d1 (0xbd2 bytes) 2024-09-15T00:09:50.886 INFO:teuthology.orchestra.run.smithi092.stdout:2971 trunc from 0xc938a00 to 0x5b46800 2024-09-15T00:09:50.895 INFO:teuthology.orchestra.run.smithi092.stdout:2977 read 0x5361200 thru 0x536e0ab (0xceac bytes) 2024-09-15T00:09:50.902 INFO:teuthology.orchestra.run.smithi092.stdout:2979 read 0x1163600 thru 0x11680e8 (0x4ae9 bytes) 2024-09-15T00:09:50.903 INFO:teuthology.orchestra.run.smithi092.stdout:2983 punch from 0x3063e00 to 0x3069000, (0x5200 bytes) 2024-09-15T00:09:50.915 INFO:teuthology.orchestra.run.smithi092.stdout:2984 write 0x9218000 thru 0x9220cf4 (0x8cf5 bytes) 2024-09-15T00:09:50.916 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:50.913+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:50.921 INFO:teuthology.orchestra.run.smithi092.stdout:2986 punch from 0x42f9e00 to 0x4301400, (0x7600 bytes) 2024-09-15T00:09:50.923 INFO:teuthology.orchestra.run.smithi092.stdout:2987 read 0x878da00 thru 0x879636e (0x896f bytes) 2024-09-15T00:09:50.939 INFO:teuthology.orchestra.run.smithi092.stdout:2990 write 0xe47cc00 thru 0xe47d3ec (0x7ed bytes) 2024-09-15T00:09:50.948 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:50.948 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:50.955 INFO:teuthology.orchestra.run.smithi092.stdout:2991 write 0xe833e00 thru 0xe83dcf3 (0x9ef4 bytes) 2024-09-15T00:09:50.957 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:50.961 INFO:teuthology.orchestra.run.smithi092.stdout:2996 trunc from 0xe83de00 to 0xe650200 2024-09-15T00:09:50.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:50.972 INFO:teuthology.orchestra.run.smithi092.stdout:3000 punch from 0xbd8b400 to 0xbd98a00, (0xd600 bytes) 2024-09-15T00:09:50.979 INFO:teuthology.orchestra.run.smithi092.stdout:3003 punch from 0xca9ca00 to 0xcaa9400, (0xca00 bytes) 2024-09-15T00:09:50.980 INFO:teuthology.orchestra.run.smithi092.stdout:3005 punch from 0x31e2e00 to 0x31eb200, (0x8400 bytes) 2024-09-15T00:09:50.980 INFO:teuthology.orchestra.run.smithi092.stdout:3006 write 0x960a00 thru 0x9699c8 (0x8fc9 bytes) 2024-09-15T00:09:50.980 INFO:teuthology.orchestra.run.smithi092.stdout:3007 punch from 0x19e7c00 to 0x19f1e00, (0xa200 bytes) 2024-09-15T00:09:50.980 INFO:teuthology.orchestra.run.smithi092.stdout:3008 read 0x4669800 thru 0x467650d (0xcd0e bytes) 2024-09-15T00:09:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:09:50.990 INFO:teuthology.orchestra.run.smithi092.stdout:3012 trunc from 0xe650200 to 0x5d8aa00 2024-09-15T00:09:51.010 INFO:teuthology.orchestra.run.smithi092.stdout:3013 read 0x1c3e400 thru 0x1c41b5d (0x375e bytes) 2024-09-15T00:09:51.017 INFO:teuthology.orchestra.run.smithi092.stdout:3014 read 0x53ccc00 thru 0x53d8c06 (0xc007 bytes) 2024-09-15T00:09:51.017 INFO:teuthology.orchestra.run.smithi092.stdout:3015 read 0x5c4200 thru 0x5d14aa (0xd2ab bytes) 2024-09-15T00:09:51.021 INFO:teuthology.orchestra.run.smithi092.stdout:3016 trunc from 0x5d8aa00 to 0x91eae00 2024-09-15T00:09:51.030 INFO:teuthology.orchestra.run.smithi092.stdout:3019 read 0x75b9000 thru 0x75bedf1 (0x5df2 bytes) 2024-09-15T00:09:51.030 INFO:teuthology.orchestra.run.smithi027.stdout:3.c 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.b 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.3 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.1f 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.1b 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.1 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.18 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.17 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.6 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.15 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.12 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:3.10 2024-09-15T00:09:51.031 INFO:teuthology.orchestra.run.smithi027.stdout:2.5 2024-09-15T00:09:51.032 INFO:teuthology.orchestra.run.smithi027.stdout:2.4 2024-09-15T00:09:51.032 INFO:teuthology.orchestra.run.smithi027.stdout:1.0 2024-09-15T00:09:51.032 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:51.029+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:51.041 INFO:teuthology.orchestra.run.smithi092.stdout:3020 trunc from 0x91eae00 to 0x48d8200 2024-09-15T00:09:51.060 INFO:teuthology.orchestra.run.smithi092.stdout:3022 clone 24 order 22 su 32768 sc 15 2024-09-15T00:09:51.110 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:09:51.113 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:51.114 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:51.118 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:51.116+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:51.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:51.127 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:51.127 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:51.146 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:51.146 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:51.164 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:51.164 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:51.181 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:51.198 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:51.198 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:51.201 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:09:51.201 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:09:51.236 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:51.236 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:51.236 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:51.237 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:51.235+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:51.244 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:51.320 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:51.318+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:51.393 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:51.393 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:51.393 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:51.394 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:51.394 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:51.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:51.420 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:51.419+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:51.510 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:51.510 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:51.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:51.510 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.482187304, 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.511 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:09:51.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.482165385999998, 2024-09-15T00:09:51.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:51.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:09:51.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.482144114, 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:09:51.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.480402854000001, 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:09:51.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.529 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.480380948000001, 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.530 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:09:51.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:09:51.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.480358855999999, 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:51.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.480327117000002, 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:09:51.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:09:51.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:09:51.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:51.543 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.480028913999998, 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.544 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:09:51.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.479995208999998, 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.550 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:09:51.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:09:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.475693136, 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.553 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:09:51.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.557 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.475368266, 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:09:51.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.562 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.474957861, 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:51.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.474902958000001, 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:09:51.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:09:51.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.474870923000001, 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.572 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:51.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.574 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.473393092999999, 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:09:51.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:09:51.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.473371378, 2024-09-15T00:09:51.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:09:51.582 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.582 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.582 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.582 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:09:51.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.582 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:09:51.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:51.586 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.473349898999999, 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:51.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:09:51.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.471336794999999, 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:09:51.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.595 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.471317154000001, 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:51.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:09:51.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.471270606000001, 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:09:51.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:09:51.605 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.606 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:51.606 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:51.606 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:51.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:51.608 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 3.c --file /var/log/ceph/exp.3.c.0 2024-09-15T00:09:51.609 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:51.605+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:51.708 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:51.706+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:51.714 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:51.714 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:51.724 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:51.808 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:51.807+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:51.866 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:51.866 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:51.866 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:51.866 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:51.866 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:51.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:51.918 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:51.917+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:51.977 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4070 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624492739000001, 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02702953, 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4070 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350999+0000", 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351003+0000", 2024-09-15T00:09:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351007+0000", 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6820000000000001e-06 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351010+0000", 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7269999999999999e-06 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377929+0000", 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026919887999999999 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377969+0000", 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9909e-05 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378028+0000", 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8578000000000001e-05 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4071 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624477858999999, 2024-09-15T00:09:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027136008, 2024-09-15T00:09:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4071 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351014+0000", 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351015+0000", 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7039999999999999e-06 2024-09-15T00:09:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351020+0000", 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351023+0000", 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378053+0000", 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027029610999999999 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378094+0000", 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1826999999999998e-05 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378149+0000", 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4815000000000003e-05 2024-09-15T00:09:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4072 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.62445318, 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027223404999999999, 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4072 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351040+0000", 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351046+0000", 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351051+0000", 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378171+0000", 2024-09-15T00:09:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027120590999999999 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378209+0000", 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7308999999999998e-05 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378261+0000", 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2753999999999997e-05 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4074 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624434285, 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027334509999999999, 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4074 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9200000000000002e-07 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351059+0000", 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.012e-06 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351065+0000", 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2129999999999996e-06 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351070+0000", 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8470000000000003e-06 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378285+0000", 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027214449000000002 2024-09-15T00:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378334+0000", 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378391+0000", 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7046000000000002e-05 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4075 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624415414000001, 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027546923000000001, 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4075 2024-09-15T00:09:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351078+0000", 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.905e-06 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351085+0000", 2024-09-15T00:09:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351090+0000", 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378423+0000", 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027333051000000001 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378523+0000", 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9959999999999998e-05 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378623+0000", 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9962000000000003e-05 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4077 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.62439273, 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027660619000000001, 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4077 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351099+0000", 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351101+0000", 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351107+0000", 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7980000000000002e-06 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351111+0000", 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378657+0000", 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027545463999999999 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378702+0000", 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5809999999999997e-05 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378759+0000", 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-15T00:09:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4078 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.62436473, 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027808405000000001, 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4078 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351127+0000", 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351129+0000", 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351135+0000", 2024-09-15T00:09:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3820000000000001e-06 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351141+0000", 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2259999999999996e-06 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378785+0000", 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027644592999999999 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378855+0000", 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0276000000000003e-05 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378935+0000", 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9361000000000001e-05 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4079 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624343093, 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027913778, 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4079 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351150+0000", 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9010000000000001e-06 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351157+0000", 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351162+0000", 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378958+0000", 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027795786999999999 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379004+0000", 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6124000000000001e-05 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379062+0000", 2024-09-15T00:09:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7630999999999998e-05 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4081 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624319411999998, 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016658, 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4081 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351174+0000", 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351181+0000", 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9070000000000001e-06 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351186+0000", 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.651e-06 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379085+0000", 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027898757999999999 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379130+0000", 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5126000000000001e-05 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379188+0000", 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8548999999999998e-05 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4082 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624299358000002, 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028118457999999999, 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4082 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351194+0000", 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7519999999999999e-06 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351200+0000", 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.801e-06 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351205+0000", 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379213+0000", 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028007539000000001 2024-09-15T00:09:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379254+0000", 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1817999999999998e-05 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379310+0000", 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5850000000000002e-05 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4083 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624277637999999, 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028218838999999999, 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4083 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351214+0000", 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351216+0000", 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351222+0000", 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351226+0000", 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379333+0000", 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028106740000000002 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379376+0000", 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3581000000000002e-05 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379432+0000", 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5868000000000001e-05 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4085 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.030 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624249133999999, 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311823, 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4085 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0699999999999998e-07 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351244+0000", 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.782e-06 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351248+0000", 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6289999999999999e-06 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351250+0000", 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379457+0000", 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028206479 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379499+0000", 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1637000000000001e-05 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379554+0000", 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5248999999999999e-05 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4086 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624237293, 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02842368, 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.035 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4086 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351259+0000", 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9520000000000002e-06 2024-09-15T00:09:52.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351262+0000", 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351265+0000", 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379576+0000", 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311228000000001 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379619+0000", 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2595000000000001e-05 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379678+0000", 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.039 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4087 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624222937999999, 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523903999999999, 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4087 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351269+0000", 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351270+0000", 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1090000000000001e-06 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351273+0000", 2024-09-15T00:09:52.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9189999999999999e-06 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351275+0000", 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379701+0000", 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028425813000000001 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379742+0000", 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0769000000000002e-05 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379792+0000", 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4088 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624206804, 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028621284, 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4088 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351285+0000", 2024-09-15T00:09:52.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351286+0000", 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.186e-06 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351289+0000", 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351292+0000", 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379815+0000", 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523211 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379857+0000", 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1807999999999997e-05 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379906+0000", 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8594999999999998e-05 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4090 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624195718999999, 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028748587999999999, 2024-09-15T00:09:52.049 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4090 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351296+0000", 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351297+0000", 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1060000000000001e-06 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351300+0000", 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9110000000000002e-06 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351302+0000", 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:09:52.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379947+0000", 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028645177000000001 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379991+0000", 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3680999999999997e-05 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380044+0000", 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2957000000000001e-05 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4091 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624186164000001, 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028864198000000001, 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.054 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4091 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351306+0000", 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351309+0000", 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351312+0000", 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380069+0000", 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028757933999999999 2024-09-15T00:09:52.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380113+0000", 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3810000000000002e-05 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380169+0000", 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5887999999999998e-05 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4092 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624176806000001, 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028973819000000001, 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4092 2024-09-15T00:09:52.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351315+0000", 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9999999999999999e-07 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351316+0000", 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.95e-07 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351318+0000", 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8629999999999999e-06 2024-09-15T00:09:52.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351321+0000", 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380195+0000", 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028873868 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380238+0000", 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2883999999999998e-05 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380288+0000", 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0096000000000003e-05 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4094 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624167187000001, 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02907964, 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4094 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:52.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351325+0000", 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.054e-06 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351328+0000", 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351331+0000", 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4679999999999999e-06 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380312+0000", 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028981587999999999 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380355+0000", 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2688000000000002e-05 2024-09-15T00:09:52.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380404+0000", 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8767999999999998e-05 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4095 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.624157799999999, 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029185068000000002, 2024-09-15T00:09:52.068 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4095 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351334+0000", 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.04e-07 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351335+0000", 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0300000000000001e-06 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351339+0000", 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351344+0000", 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0560000000000004e-06 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380428+0000", 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02908444 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380470+0000", 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1338000000000003e-05 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380518+0000", 2024-09-15T00:09:52.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8724999999999999e-05 2024-09-15T00:09:52.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.073 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:52.074 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone23 from 0x48d8200 (overlap 0xc1be00) to 0x9afe00 2024-09-15T00:09:52.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:52.075 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:52.072+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:52.175 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:52.174+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:52.220 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:52.220 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:52.220 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:52.220 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:52.229 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:52.276 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:52.275+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:52.371 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-15T00:09:52.376 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:52.376 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:52.377 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:52.377 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:52.377 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:52.377 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:52.376+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:52.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:52.478 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:52.476+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:52.498 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:52.498 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3427 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208725033, 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027724631, 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3427 2024-09-15T00:09:52.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287582+0000", 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2599999999999998e-07 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287584+0000", 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8759999999999999e-06 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287589+0000", 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-15T00:09:52.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287594+0000", 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.357e-06 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315217+0000", 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027622785 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315257+0000", 2024-09-15T00:09:52.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0228000000000002e-05 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315306+0000", 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9228999999999999e-05 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3428 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:52.503 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208707027999999, 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027928562000000001, 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3428 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287600+0000", 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287602+0000", 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287608+0000", 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287613+0000", 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8740000000000003e-06 2024-09-15T00:09:52.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315346+0000", 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027733099000000001 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315431+0000", 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4350000000000004e-05 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315528+0000", 2024-09-15T00:09:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7374000000000007e-05 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3429 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208687718, 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028028630999999998, 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3429 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287621+0000", 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287627+0000", 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8440000000000003e-06 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287631+0000", 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5009999999999998e-06 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315556+0000", 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027925241999999999 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315595+0000", 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8512e-05 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315647+0000", 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2318000000000003e-05 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.512 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3430 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208669643999997, 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028122114, 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3430 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:09:52.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287644+0000", 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4219999999999997e-06 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287650+0000", 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0920000000000003e-06 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287654+0000", 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4279999999999998e-06 2024-09-15T00:09:52.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315671+0000", 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016592 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315709+0000", 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8231e-05 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315759+0000", 2024-09-15T00:09:52.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9926999999999998e-05 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3431 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208646104000003, 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02831842, 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3431 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287661+0000", 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287663+0000", 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9570000000000001e-06 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287668+0000", 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287673+0000", 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315799+0000", 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028125741999999999 2024-09-15T00:09:52.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315891+0000", 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2701000000000006e-05 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315979+0000", 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7287000000000006e-05 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3432 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208627720000003, 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028430624000000002, 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3432 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.522 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:52.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287681+0000", 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8530000000000001e-06 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287686+0000", 2024-09-15T00:09:52.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4659999999999998e-06 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287691+0000", 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2230000000000001e-06 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316006+0000", 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028315344999999999 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316054+0000", 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8004999999999999e-05 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316109+0000", 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5374999999999998e-05 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.526 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3433 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208609963999997, 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028528593000000001, 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3433 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287697+0000", 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287699+0000", 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8509999999999999e-06 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287704+0000", 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287710+0000", 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316124+0000", 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028414466999999999 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316171+0000", 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6783999999999998e-05 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316225+0000", 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3891999999999998e-05 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3434 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208590579000003, 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028632801999999999, 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3434 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-07 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287718+0000", 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287724+0000", 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8390000000000003e-06 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287729+0000", 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8389999999999997e-06 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316250+0000", 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028521135 2024-09-15T00:09:52.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316295+0000", 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4540999999999999e-05 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316349+0000", 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4083999999999999e-05 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3435 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208559903000001, 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028723243999999998, 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3435 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287747+0000", 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4900000000000002e-07 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287748+0000", 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4190000000000001e-06 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287752+0000", 2024-09-15T00:09:52.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7900000000000001e-06 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287756+0000", 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1609999999999996e-06 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316375+0000", 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028618932 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316417+0000", 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316470+0000", 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3436 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208544855, 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028827854, 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3436 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:52.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287766+0000", 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287771+0000", 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5140000000000001e-06 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287776+0000", 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316494+0000", 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028718144000000001 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316538+0000", 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4129999999999999e-05 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316589+0000", 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1548999999999997e-05 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3437 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208526962999997, 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028972188999999999, 2024-09-15T00:09:52.545 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3437 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287780+0000", 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287782+0000", 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8419999999999999e-06 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287787+0000", 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3059999999999997e-06 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287791+0000", 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9920000000000001e-06 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316615+0000", 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028824487999999999 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316668+0000", 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2883999999999997e-05 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316752+0000", 2024-09-15T00:09:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3308999999999998e-05 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3438 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208509921000001, 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029124331999999999, 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3438 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287797+0000", 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287799+0000", 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287804+0000", 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7069999999999997e-06 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287808+0000", 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0720000000000001e-06 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316799+0000", 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028990241999999999 2024-09-15T00:09:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316845+0000", 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5837000000000002e-05 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316921+0000", 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6255999999999996e-05 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3439 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208492382000003, 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029290665, 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3439 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287816+0000", 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6980000000000001e-06 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287824+0000", 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2500000000000006e-06 2024-09-15T00:09:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287829+0000", 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316967+0000", 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029138319999999999 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317047+0000", 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0081999999999997e-05 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317105+0000", 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7497999999999997e-05 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3440 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208471322999998, 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029395451999999999, 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3440 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:09:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287837+0000", 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287843+0000", 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6799999999999998e-06 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287848+0000", 2024-09-15T00:09:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317131+0000", 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029283337999999999 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317176+0000", 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5176999999999998e-05 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317231+0000", 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4120999999999999e-05 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3441 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208453259000002, 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029496321999999998, 2024-09-15T00:09:52.564 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3441 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287855+0000", 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7099999999999999e-06 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287861+0000", 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287866+0000", 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317252+0000", 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029386051999999999 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317297+0000", 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5164999999999999e-05 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317349+0000", 2024-09-15T00:09:52.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2534000000000002e-05 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3442 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208433993, 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029594697999999999, 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3442 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287873+0000", 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287879+0000", 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-15T00:09:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287885+0000", 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.755e-06 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287890+0000", 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317375+0000", 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029485457 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317416+0000", 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1217999999999997e-05 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317467+0000", 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0819999999999998e-05 2024-09-15T00:09:52.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3443 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208410653999998, 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029684676, 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3443 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287898+0000", 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287904+0000", 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8860000000000001e-06 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287908+0000", 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5650000000000003e-06 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317488+0000", 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029579437 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317530+0000", 2024-09-15T00:09:52.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1900999999999997e-05 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317580+0000", 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0603000000000003e-05 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3444 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208385436, 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029772823, 2024-09-15T00:09:52.578 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3444 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287931+0000", 2024-09-15T00:09:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0008e-05 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287938+0000", 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287943+0000", 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317603+0000", 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029659913999999999 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317646+0000", 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2367999999999998e-05 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317694+0000", 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8201999999999997e-05 2024-09-15T00:09:52.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3445 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208356098000003, 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029863565000000002, 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3445 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287951+0000", 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287953+0000", 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.765e-06 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287958+0000", 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999996e-06 2024-09-15T00:09:52.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287963+0000", 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.634e-06 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317714+0000", 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029751353000000001 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317755+0000", 2024-09-15T00:09:52.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0342000000000001e-05 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317814+0000", 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9188999999999999e-05 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3446 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.208337800000002, 2024-09-15T00:09:52.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029967884, 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3446 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:52.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287971+0000", 2024-09-15T00:09:52.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287977+0000", 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4330000000000003e-06 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287981+0000", 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3989999999999997e-06 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317837+0000", 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029855942 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317878+0000", 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467000000000001e-05 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317936+0000", 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8100999999999998e-05 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.592 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.592 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:52.593 INFO:teuthology.orchestra.run.smithi092.stdout:3023 write 0x888a800 thru 0x88916f7 (0x6ef8 bytes) 2024-09-15T00:09:52.593 INFO:teuthology.orchestra.run.smithi092.stdout:3027 punch from 0x11e2600 to 0x11eda00, (0xb400 bytes) 2024-09-15T00:09:52.593 INFO:teuthology.orchestra.run.smithi092.stdout:3029 trunc from 0x8891800 to 0x1deb200 2024-09-15T00:09:52.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:52.594 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:52.591+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:52.694 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:52.693+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:52.698 INFO:teuthology.orchestra.run.smithi092.stdout:3032 trunc from 0x1deb200 to 0x10ad000 2024-09-15T00:09:52.713 INFO:teuthology.orchestra.run.smithi092.stdout:3033 read 0x46b200 thru 0x47b07e (0xfe7f bytes) 2024-09-15T00:09:52.714 INFO:teuthology.orchestra.run.smithi092.stdout:3036 read 0xedbc00 thru 0xee7ca3 (0xc0a4 bytes) 2024-09-15T00:09:52.715 INFO:teuthology.orchestra.run.smithi092.stdout:3038 read 0x9e2200 thru 0x9e4b3c (0x293d bytes) 2024-09-15T00:09:52.715 INFO:teuthology.orchestra.run.smithi092.stdout:3040 read 0x461400 thru 0x466a1c (0x561d bytes) 2024-09-15T00:09:52.716 INFO:teuthology.orchestra.run.smithi092.stdout:3041 read 0x401e00 thru 0x40c5a4 (0xa7a5 bytes) 2024-09-15T00:09:52.726 INFO:teuthology.orchestra.run.smithi092.stdout:3042 write 0x1e80c00 thru 0x1e89efe (0x92ff bytes) 2024-09-15T00:09:52.747 INFO:teuthology.orchestra.run.smithi092.stdout:3043 write 0xae66800 thru 0xae6e0fc (0x78fd bytes) 2024-09-15T00:09:52.755 INFO:teuthology.orchestra.run.smithi092.stdout:3044 read 0x313f400 thru 0x314b2e0 (0xbee1 bytes) 2024-09-15T00:09:52.757 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5265.0:2447 3.10 3:0995ba94:::rbd_data.112986cdeff1.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-15T00:09:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:52.755135+0000", 2024-09-15T00:09:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00093473700000000005, 2024-09-15T00:09:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000952465, 2024-09-15T00:09:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5265", 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/562464505", 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2447 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755135+0000", 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755135+0000", 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755136+0000", 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755137+0000", 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032e-06 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755140+0000", 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-06 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755143+0000", 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6359999999999998e-06 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755998+0000", 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085488399999999998 2024-09-15T00:09:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.756071+0000", 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3391000000000003e-05 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5265.0:2448 3.10 3.295da990 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:52.755147+0000", 2024-09-15T00:09:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00092347999999999996, 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098028500000000001, 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5265", 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/562464505", 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2448 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755147+0000", 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755147+0000", 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755147+0000", 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2000000000000001e-07 2024-09-15T00:09:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755148+0000", 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3179999999999999e-06 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755151+0000", 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8899999999999999e-06 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755154+0000", 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5919999999999999e-06 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5265.0:2449 3.10 3.295da990 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:52.755157+0000", 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00091281000000000001, 2024-09-15T00:09:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099874399999999998, 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5265", 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/562464505", 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2449 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755157+0000", 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755157+0000", 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755157+0000", 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755159+0000", 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1209999999999999e-06 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755161+0000", 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8789999999999999e-06 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755164+0000", 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5620000000000002e-06 2024-09-15T00:09:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5265.0:2450 3.10 3.295da990 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:52.755177+0000", 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00089276700000000004, 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010054059999999999, 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-15T00:09:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5265", 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/562464505", 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2450 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755177+0000", 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755177+0000", 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755178+0000", 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.96e-07 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755179+0000", 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.86e-06 2024-09-15T00:09:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755184+0000", 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4519999999999999e-06 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:52.755188+0000", 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.687e-06 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:09:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 4 2024-09-15T00:09:52.773 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:52.773 INFO:teuthology.orchestra.run.smithi092.stdout:3045 trunc from 0xae6e200 to 0x4c42e00 2024-09-15T00:09:52.774 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:52.803 INFO:teuthology.orchestra.run.smithi092.stdout:3046 write 0x6657a00 thru 0x6665900 (0xdf01 bytes) 2024-09-15T00:09:52.803 INFO:teuthology.orchestra.run.smithi092.stdout:3047 write 0x3acba00 thru 0x3accc6a (0x126b bytes) 2024-09-15T00:09:52.804 INFO:teuthology.orchestra.run.smithi092.stdout:3048 punch from 0x64a0a00 to 0x64ae200, (0xd800 bytes) 2024-09-15T00:09:52.808 INFO:teuthology.orchestra.run.smithi092.stdout:3049 read 0x2662e00 thru 0x266a7e8 (0x79e9 bytes) 2024-09-15T00:09:52.808 INFO:teuthology.orchestra.run.smithi092.stdout:3050 punch from 0x2925400 to 0x2925a00, (0x600 bytes) 2024-09-15T00:09:52.808 INFO:teuthology.orchestra.run.smithi092.stdout:3051 write 0x19de600 thru 0x19ed3f9 (0xedfa bytes) 2024-09-15T00:09:52.809 INFO:teuthology.orchestra.run.smithi092.stdout:3053 read 0x289c800 thru 0x28a8a09 (0xc20a bytes) 2024-09-15T00:09:52.817 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:52.816+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:52.833 INFO:teuthology.orchestra.run.smithi092.stdout:3054 write 0xe218800 thru 0xe227b0d (0xf30e bytes) 2024-09-15T00:09:52.834 INFO:teuthology.orchestra.run.smithi092.stdout:3055 trunc from 0xe227c00 to 0xd8de600 2024-09-15T00:09:52.849 INFO:teuthology.orchestra.run.smithi092.stdout:3057 write 0x665ce00 thru 0x665d51b (0x71c bytes) 2024-09-15T00:09:52.869 INFO:teuthology.orchestra.run.smithi092.stdout:3058 trunc from 0xd8de600 to 0x10c9200 2024-09-15T00:09:52.890 INFO:teuthology.orchestra.run.smithi092.stdout:3059 read 0x8c4400 thru 0x8cf0b8 (0xacb9 bytes) 2024-09-15T00:09:52.891 INFO:teuthology.orchestra.run.smithi092.stdout:3060 read 0x3dc800 thru 0x3ea1f6 (0xd9f7 bytes) 2024-09-15T00:09:52.892 INFO:teuthology.orchestra.run.smithi092.stdout:3062 read 0x664200 thru 0x672509 (0xe30a bytes) 2024-09-15T00:09:52.895 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:52.895 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:52.895 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:52.895 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:52.895 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:52.896 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:52.894+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:52.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:52.914 INFO:teuthology.orchestra.run.smithi092.stdout:3067 write 0xc5e8600 thru 0xc5f7b24 (0xf525 bytes) 2024-09-15T00:09:52.916 INFO:teuthology.orchestra.run.smithi092.stdout:3068 trunc from 0xc5f7c00 to 0xbd6cc00 2024-09-15T00:09:52.932 INFO:teuthology.orchestra.run.smithi092.stdout:3070 trunc from 0xbd6cc00 to 0x8c6b600 2024-09-15T00:09:52.944 INFO:teuthology.orchestra.run.smithi092.stdout:3071 read 0x62dfe00 thru 0x62e3623 (0x3824 bytes) 2024-09-15T00:09:52.946 INFO:teuthology.orchestra.run.smithi092.stdout:3072 write 0x75aa000 thru 0x75afaf5 (0x5af6 bytes) 2024-09-15T00:09:52.946 INFO:teuthology.orchestra.run.smithi092.stdout:3079 punch from 0x16c1400 to 0x16c4600, (0x3200 bytes) 2024-09-15T00:09:52.947 INFO:teuthology.orchestra.run.smithi092.stdout:3081 punch from 0x65dce00 to 0x65e1c00, (0x4e00 bytes) 2024-09-15T00:09:52.947 INFO:teuthology.orchestra.run.smithi092.stdout:3083 punch from 0x549400 to 0x555000, (0xbc00 bytes) 2024-09-15T00:09:52.948 INFO:teuthology.orchestra.run.smithi092.stdout:3084 trunc from 0x8c6b600 to 0x8121a00 2024-09-15T00:09:52.960 INFO:teuthology.orchestra.run.smithi092.stdout:3085 read 0x2249400 thru 0x22545db (0xb1dc bytes) 2024-09-15T00:09:52.969 INFO:teuthology.orchestra.run.smithi092.stdout:3087 trunc from 0x8121a00 to 0xe4cee00 2024-09-15T00:09:52.979 INFO:teuthology.orchestra.run.smithi092.stdout:3088 read 0x75dac00 thru 0x75e4dda (0xa1db bytes) 2024-09-15T00:09:52.980 INFO:teuthology.orchestra.run.smithi092.stdout:3090 read 0x76a4800 thru 0x76ac77b (0x7f7c bytes) 2024-09-15T00:09:52.980 INFO:teuthology.orchestra.run.smithi092.stdout:3092 read 0xd545800 thru 0xd5502fa (0xaafb bytes) 2024-09-15T00:09:52.980 INFO:teuthology.orchestra.run.smithi092.stdout:3094 read 0x61b6000 thru 0x61bafc6 (0x4fc7 bytes) 2024-09-15T00:09:52.981 INFO:teuthology.orchestra.run.smithi092.stdout:3095 write 0x796e600 thru 0x79798c5 (0xb2c6 bytes) 2024-09-15T00:09:52.981 INFO:teuthology.orchestra.run.smithi092.stdout:3097 read 0xce4c00 thru 0xce998e (0x4d8f bytes) 2024-09-15T00:09:52.981 INFO:teuthology.orchestra.run.smithi092.stdout:3098 write 0xdd89a00 thru 0xdd95432 (0xba33 bytes) 2024-09-15T00:09:52.981 INFO:teuthology.orchestra.run.smithi092.stdout:3099 punch from 0x689d400 to 0x68a4800, (0x7400 bytes) 2024-09-15T00:09:52.982 INFO:teuthology.orchestra.run.smithi092.stdout:3100 write 0x916b400 thru 0x91749cf (0x95d0 bytes) 2024-09-15T00:09:52.982 INFO:teuthology.orchestra.run.smithi092.stdout:3102 read 0x77b7200 thru 0x77baea3 (0x3ca4 bytes) 2024-09-15T00:09:52.982 INFO:teuthology.orchestra.run.smithi092.stdout:3107 write 0x5cc8400 thru 0x5cd3728 (0xb329 bytes) 2024-09-15T00:09:52.984 INFO:teuthology.orchestra.run.smithi092.stdout:3108 read 0x4fda000 thru 0x4fe4c6b (0xac6c bytes) 2024-09-15T00:09:52.985 INFO:teuthology.orchestra.run.smithi092.stdout:3109 read 0x657da00 thru 0x65895b2 (0xbbb3 bytes) 2024-09-15T00:09:52.985 INFO:teuthology.orchestra.run.smithi092.stdout:3111 read 0x366f200 thru 0x36742d3 (0x50d4 bytes) 2024-09-15T00:09:52.985 INFO:teuthology.orchestra.run.smithi092.stdout:3113 write 0xa407800 thru 0xa412583 (0xad84 bytes) 2024-09-15T00:09:52.987 INFO:teuthology.orchestra.run.smithi092.stdout:3114 write 0xdff9800 thru 0xdffbbac (0x23ad bytes) 2024-09-15T00:09:52.987 INFO:teuthology.orchestra.run.smithi092.stdout:3116 read 0x7807c00 thru 0x7815186 (0xd587 bytes) 2024-09-15T00:09:52.989 INFO:teuthology.orchestra.run.smithi092.stdout:3118 trunc from 0xe4cee00 to 0xcef8800 2024-09-15T00:09:52.996 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:52.995+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:53.010 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:09:53.013 INFO:teuthology.orchestra.run.smithi092.stdout:3119 clone 25 order 24 su 131072 sc 14 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1142 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.177899457000002, 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045190363999999997, 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.063 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1142 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:53.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882756+0000", 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882773+0000", 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6872999999999999e-05 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882780+0000", 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4240000000000004e-06 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882787+0000", 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8510000000000001e-06 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927799+0000", 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045011974000000003 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927843+0000", 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4270999999999997e-05 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927946+0000", 2024-09-15T00:09:53.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102581 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.177850765000002, 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:09:53.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:53.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:09:53.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:09:53.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.076 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.076 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:53.076 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.177812135000003, 2024-09-15T00:09:53.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:09:53.076 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.076 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.076 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:53.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.080 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.177786198, 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:53.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:09:53.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.177770129999999, 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:53.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:09:53.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.089 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.177629035000002, 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.090 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.17685728, 2024-09-15T00:09:53.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:09:53.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.098 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.176772249000003, 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.099 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:09:53.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.176737232999997, 2024-09-15T00:09:53.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:09:53.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.17671163, 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:09:53.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:09:53.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.176537363999998, 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:09:53.112 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:09:53.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.116 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.176306224999998, 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:09:53.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1157 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.175524787000001, 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045184281, 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.121 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1157 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885132+0000", 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885139+0000", 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0350000000000004e-06 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885153+0000", 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3515e-05 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930177+0000", 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045024495999999997 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930249+0000", 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2072999999999999e-05 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930314+0000", 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4826999999999994e-05 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.175494280999999, 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.126 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:09:53.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.175414541000002, 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:09:53.130 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:53.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:09:53.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:09:53.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.175209516000002, 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:53.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.174997982999997, 2024-09-15T00:09:53.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:09:53.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.143 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.172441976000002, 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:09:53.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.170458869999997, 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:09:53.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:53.153 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.170439299000002, 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:09:53.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:09:53.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:53.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:09:53.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:09:53.158 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.158 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.158 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.158 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:53.158 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:53.158 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:53.159 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:53.157+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:53.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:53.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:53.260 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:53.258+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:53.270 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:53.270 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:53.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:53.316 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:09:53.361 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:53.359+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:53.388 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:53.388 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:53.388 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:53.388 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:53.389 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:53.397 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:53.462 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:53.460+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:53.533 INFO:teuthology.orchestra.run.smithi027.stderr:Exporting 3.c info 3.c( v 55'276 (0'0,55'276] local-lis/les=20/21 n=22 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-15T00:09:53.535 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:53.541 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:3005b819:::rbd_object_map.11291357fc5b:head# 2024-09-15T00:09:53.542 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:302ce972:::rbd_data.1129ebdcc4e7.000000000000000f:head# 2024-09-15T00:09:53.545 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:307e958b:::rbd_data.11296e862a2c.000000000000005f:c# 2024-09-15T00:09:53.546 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:307e958b:::rbd_data.11296e862a2c.000000000000005f:head# 2024-09-15T00:09:53.547 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:30d5d670:::rbd_data.11293dff0670.0000000000000007:e# 2024-09-15T00:09:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:30d5d670:::rbd_data.11293dff0670.0000000000000007:head# 2024-09-15T00:09:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:3121db20:::rbd_data.1129aa143e51.000000000000005d:5# 2024-09-15T00:09:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:3121db20:::rbd_data.1129aa143e51.000000000000005d:head# 2024-09-15T00:09:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:32606b9a:::rbd_object_map.1129abeafde8:head# 2024-09-15T00:09:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:33bd27dc:::rbd_data.11293dff0670.0000000000000024:e# 2024-09-15T00:09:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:33bd27dc:::rbd_data.11293dff0670.0000000000000024:head# 2024-09-15T00:09:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:340739f1:::rbd_data.112927a1b33d.0000000000000008:head# 2024-09-15T00:09:53.550 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:53.550 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:53.550 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:34ae4a59:::rbd_data.112910301a74.000000000000000f:head# 2024-09-15T00:09:53.552 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:34b47bce:::rbd_data.1129aa143e51.000000000000005c:5# 2024-09-15T00:09:53.552 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:34b47bce:::rbd_data.1129aa143e51.000000000000005c:head# 2024-09-15T00:09:53.552 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:34b73b99:::rbd_data.11298f711a6e.000000000000000b:head# 2024-09-15T00:09:53.553 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:35432b3e:::rbd_data.11291ce3fc54.0000000000000017:6# 2024-09-15T00:09:53.553 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:35432b3e:::rbd_data.11291ce3fc54.0000000000000017:head# 2024-09-15T00:09:53.553 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:355b8a9d:::rbd_data.1129aa143e51.0000000000000067:head# 2024-09-15T00:09:53.553 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:3560610b:::rbd_data.112981949474.0000000000000043:head# 2024-09-15T00:09:53.553 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:37551d32:::rbd_data.11296e862a2c.000000000000005b:c# 2024-09-15T00:09:53.553 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:53.554 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:53.554 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:37551d32:::rbd_data.11296e862a2c.000000000000005b:head# 2024-09-15T00:09:53.554 INFO:teuthology.orchestra.run.smithi027.stderr:Export successful 2024-09-15T00:09:53.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:53.562 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:53.561+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:53.571 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:53.589 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:53.589 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:53.622 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:53.622 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:53.625 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:09:53.626 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:09:53.663 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:53.661+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:53.679 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:53.679 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:53.688 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:53.763 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:53.762+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:53.795 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:53.795 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:53.803 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:53.864 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:53.863+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:53.910 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:53.910 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:53.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:53.966 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:53.964+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:54.004 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:09:54.032 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:54.032 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:54.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:54.066 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:54.065+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:54.077 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 3.c 2024-09-15T00:09:54.084 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone24 from 0xcef8800 (overlap 0x10ad000) to 0x856800 2024-09-15T00:09:54.167 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:54.166+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:54.195 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:54.195 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:54.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:54.268 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:54.267+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:54.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-15T00:09:54.307 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:54.308 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:54.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:54.369 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:54.368+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:54.423 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:54.423 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:54.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:54.470 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:54.469+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:54.515 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-15T00:09:54.571 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:54.570+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:54.577 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:54.577 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:54.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:54.624 INFO:teuthology.orchestra.run.smithi092.stdout:3122 write 0xde29800 thru 0xde38441 (0xec42 bytes) 2024-09-15T00:09:54.625 INFO:teuthology.orchestra.run.smithi092.stdout:3124 write 0x7d20800 thru 0x7d22ad1 (0x22d2 bytes) 2024-09-15T00:09:54.625 INFO:teuthology.orchestra.run.smithi092.stdout:3126 punch from 0x5a72800 to 0x5a7f800, (0xd000 bytes) 2024-09-15T00:09:54.632 INFO:teuthology.orchestra.run.smithi092.stdout:3127 write 0xc769200 thru 0xc77843b (0xf23c bytes) 2024-09-15T00:09:54.633 INFO:teuthology.orchestra.run.smithi092.stdout:3129 flatten 2024-09-15T00:09:54.672 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:54.671+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:54.735 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:54.736 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:54.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:54.792 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:54.791+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:54.836 INFO:teuthology.orchestra.run.smithi092.stdout:3130 read 0x97e3000 thru 0x97e536c (0x236d bytes) 2024-09-15T00:09:54.836 INFO:teuthology.orchestra.run.smithi092.stdout:3132 punch from 0x3e78800 to 0x3e79e00, (0x1600 bytes) 2024-09-15T00:09:54.836 INFO:teuthology.orchestra.run.smithi092.stdout:3133 write 0x2d8c00 thru 0x2de645 (0x5a46 bytes) 2024-09-15T00:09:54.837 INFO:teuthology.orchestra.run.smithi092.stdout:3135 write 0x1f6ce00 thru 0x1f78f4e (0xc14f bytes) 2024-09-15T00:09:54.837 INFO:teuthology.orchestra.run.smithi092.stdout:3136 read 0x20cc800 thru 0x20d16e7 (0x4ee8 bytes) 2024-09-15T00:09:54.838 INFO:teuthology.orchestra.run.smithi092.stdout:3138 read 0xb63f600 thru 0xb6471e3 (0x7be4 bytes) 2024-09-15T00:09:54.838 INFO:teuthology.orchestra.run.smithi092.stdout:3139 write 0x15dea00 thru 0x15dfbd7 (0x11d8 bytes) 2024-09-15T00:09:54.839 INFO:teuthology.orchestra.run.smithi092.stdout:3140 write 0xd6ea400 thru 0xd6f83dc (0xdfdd bytes) 2024-09-15T00:09:54.839 INFO:teuthology.orchestra.run.smithi092.stdout:3141 write 0x4d8b600 thru 0x4d911b5 (0x5bb6 bytes) 2024-09-15T00:09:54.841 INFO:teuthology.orchestra.run.smithi092.stdout:3142 trunc from 0xde38600 to 0xc7dc400 2024-09-15T00:09:54.865 INFO:teuthology.orchestra.run.smithi092.stdout:3143 read 0xac7d800 thru 0xac84341 (0x6b42 bytes) 2024-09-15T00:09:54.869 INFO:teuthology.orchestra.run.smithi092.stdout:3145 read 0x1c65400 thru 0x1c6e3f7 (0x8ff8 bytes) 2024-09-15T00:09:54.874 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:54.872+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:54.885 INFO:teuthology.orchestra.run.smithi092.stdout:3146 trunc from 0xc7dc400 to 0xc7600 2024-09-15T00:09:54.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:54.935 INFO:teuthology.orchestra.run.smithi092.stdout:3147 read 0x19400 thru 0x20f09 (0x7b0a bytes) 2024-09-15T00:09:54.940 INFO:teuthology.orchestra.run.smithi092.stdout:3150 punch from 0x5f800 to 0x64800, (0x5000 bytes) 2024-09-15T00:09:54.947 INFO:teuthology.orchestra.run.smithi092.stdout:3151 trunc from 0xc7600 to 0x54e5400 2024-09-15T00:09:54.955 INFO:teuthology.orchestra.run.smithi092.stdout:3154 read 0x1140800 thru 0x114e6ca (0xdecb bytes) 2024-09-15T00:09:54.955 INFO:teuthology.orchestra.run.smithi092.stdout:3156 write 0x31d9800 thru 0x31e6e7c (0xd67d bytes) 2024-09-15T00:09:54.960 INFO:teuthology.orchestra.run.smithi092.stdout:3157 trunc from 0x54e5400 to 0x2011400 2024-09-15T00:09:54.975 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:54.974+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:54.989 INFO:teuthology.orchestra.run.smithi092.stdout:3158 write 0x5ad2c00 thru 0x5ad5930 (0x2d31 bytes) 2024-09-15T00:09:54.999 INFO:teuthology.orchestra.run.smithi092.stdout:3159 trunc from 0x5ad5a00 to 0xe063400 2024-09-15T00:09:55.012 INFO:teuthology.orchestra.run.smithi092.stdout:3165 read 0x16b8800 thru 0x16c3a69 (0xb26a bytes) 2024-09-15T00:09:55.020 INFO:teuthology.orchestra.run.smithi092.stdout:3166 write 0x3fe1400 thru 0x3fe7149 (0x5d4a bytes) 2024-09-15T00:09:55.023 INFO:teuthology.orchestra.run.smithi092.stdout:3168 trunc from 0xe063400 to 0xc10f200 2024-09-15T00:09:55.036 INFO:teuthology.orchestra.run.smithi092.stdout:3170 punch from 0xaea0a00 to 0xaea3c00, (0x3200 bytes) 2024-09-15T00:09:55.041 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:55.041 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:55.044 INFO:teuthology.orchestra.run.smithi092.stdout:3172 read 0x9c8a000 thru 0x9c8c6e1 (0x26e2 bytes) 2024-09-15T00:09:55.044 INFO:teuthology.orchestra.run.smithi092.stdout:3180 write 0xaf9ba00 thru 0xafaa6d8 (0xecd9 bytes) 2024-09-15T00:09:55.045 INFO:teuthology.orchestra.run.smithi092.stdout:3181 read 0x272e000 thru 0x2735d8a (0x7d8b bytes) 2024-09-15T00:09:55.046 INFO:teuthology.orchestra.run.smithi092.stdout:3182 read 0x3f2c00 thru 0x4026e1 (0xfae2 bytes) 2024-09-15T00:09:55.046 INFO:teuthology.orchestra.run.smithi092.stdout:3186 write 0x5c4e400 thru 0x5c550ba (0x6cbb bytes) 2024-09-15T00:09:55.047 INFO:teuthology.orchestra.run.smithi092.stdout:3187 write 0x9787800 thru 0x978c5d8 (0x4dd9 bytes) 2024-09-15T00:09:55.047 INFO:teuthology.orchestra.run.smithi092.stdout:3189 read 0x5f14600 thru 0x5f21e7f (0xd880 bytes) 2024-09-15T00:09:55.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:55.058 INFO:teuthology.orchestra.run.smithi092.stdout:3190 trunc from 0xc10f200 to 0x2627400 2024-09-15T00:09:55.085 INFO:teuthology.orchestra.run.smithi092.stdout:3191 trunc from 0x2627400 to 0x73fce00 2024-09-15T00:09:55.094 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:55.093+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:55.102 INFO:teuthology.orchestra.run.smithi092.stdout:3195 punch from 0x2c77200 to 0x2c85c00, (0xea00 bytes) 2024-09-15T00:09:55.103 INFO:teuthology.orchestra.run.smithi092.stdout:3196 read 0x6258800 thru 0x6258e8a (0x68b bytes) 2024-09-15T00:09:55.103 INFO:teuthology.orchestra.run.smithi092.stdout:3197 read 0x4d06600 thru 0x4d0e0ea (0x7aeb bytes) 2024-09-15T00:09:55.116 INFO:teuthology.orchestra.run.smithi092.stdout:3198 write 0x9ca2200 thru 0x9cadb6f (0xb970 bytes) 2024-09-15T00:09:55.125 INFO:teuthology.orchestra.run.smithi092.stdout:3199 trunc from 0x9cadc00 to 0x3a97800 2024-09-15T00:09:55.145 INFO:teuthology.orchestra.run.smithi092.stdout:3201 trunc from 0x3a97800 to 0x8b5b200 2024-09-15T00:09:55.154 INFO:teuthology.orchestra.run.smithi092.stdout:3202 punch from 0x16b5800 to 0x16b8e00, (0x3600 bytes) 2024-09-15T00:09:55.167 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:55.167 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:55.169 INFO:teuthology.orchestra.run.smithi092.stdout:3203 write 0xddd0200 thru 0xdddaf64 (0xad65 bytes) 2024-09-15T00:09:55.170 INFO:teuthology.orchestra.run.smithi092.stdout:3204 read 0xd601200 thru 0xd61008d (0xee8e bytes) 2024-09-15T00:09:55.171 INFO:teuthology.orchestra.run.smithi092.stdout:3205 read 0x9620800 thru 0x962db68 (0xd369 bytes) 2024-09-15T00:09:55.171 INFO:teuthology.orchestra.run.smithi092.stdout:3207 write 0x86c3a00 thru 0x86c987b (0x5e7c bytes) 2024-09-15T00:09:55.172 INFO:teuthology.orchestra.run.smithi092.stdout:3208 write 0x2db1e00 thru 0x2db8b9a (0x6d9b bytes) 2024-09-15T00:09:55.173 INFO:teuthology.orchestra.run.smithi092.stdout:3211 read 0x8b4e400 thru 0x8b54b0d (0x670e bytes) 2024-09-15T00:09:55.173 INFO:teuthology.orchestra.run.smithi092.stdout:3212 write 0x183d000 thru 0x1840ade (0x3adf bytes) 2024-09-15T00:09:55.174 INFO:teuthology.orchestra.run.smithi092.stdout:3213 read 0xb9ade00 thru 0xb9b4637 (0x6838 bytes) 2024-09-15T00:09:55.174 INFO:teuthology.orchestra.run.smithi092.stdout:3218 read 0xc884000 thru 0xc891315 (0xd316 bytes) 2024-09-15T00:09:55.175 INFO:teuthology.orchestra.run.smithi092.stdout:3219 punch from 0x9c04400 to 0x9c06000, (0x1c00 bytes) 2024-09-15T00:09:55.175 INFO:teuthology.orchestra.run.smithi092.stdout:3220 read 0x52c1000 thru 0x52cdd70 (0xcd71 bytes) 2024-09-15T00:09:55.176 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:55.174+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:55.176 INFO:teuthology.orchestra.run.smithi092.stdout:3221 read 0xb69ee00 thru 0xb6a129f (0x24a0 bytes) 2024-09-15T00:09:55.177 INFO:teuthology.orchestra.run.smithi092.stdout:3222 punch from 0x5abca00 to 0x5acc400, (0xfa00 bytes) 2024-09-15T00:09:55.177 INFO:teuthology.orchestra.run.smithi092.stdout:3226 write 0x1551400 thru 0x155b09e (0x9c9f bytes) 2024-09-15T00:09:55.178 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:55.179 INFO:teuthology.orchestra.run.smithi092.stdout:3228 punch from 0x5331200 to 0x5338a00, (0x7800 bytes) 2024-09-15T00:09:55.179 INFO:teuthology.orchestra.run.smithi092.stdout:3229 read 0x437d200 thru 0x4388758 (0xb559 bytes) 2024-09-15T00:09:55.180 INFO:teuthology.orchestra.run.smithi092.stdout:3230 write 0x50a4a00 thru 0x50ad546 (0x8b47 bytes) 2024-09-15T00:09:55.181 INFO:teuthology.orchestra.run.smithi092.stdout:3233 read 0x9d8e200 thru 0x9d8f616 (0x1417 bytes) 2024-09-15T00:09:55.181 INFO:teuthology.orchestra.run.smithi092.stdout:3234 read 0x6f51200 thru 0x6f5bd68 (0xab69 bytes) 2024-09-15T00:09:55.182 INFO:teuthology.orchestra.run.smithi092.stdout:3235 write 0x6653600 thru 0x665aef1 (0x78f2 bytes) 2024-09-15T00:09:55.184 INFO:teuthology.orchestra.run.smithi092.stdout:3237 trunc from 0xdddb000 to 0xc79f000 2024-09-15T00:09:55.196 INFO:teuthology.orchestra.run.smithi092.stdout:3238 read 0xa0caa00 thru 0xa0cad87 (0x388 bytes) 2024-09-15T00:09:55.197 INFO:teuthology.orchestra.run.smithi092.stdout:3239 read 0x52bae00 thru 0x52c2baf (0x7db0 bytes) 2024-09-15T00:09:55.197 INFO:teuthology.orchestra.run.smithi092.stdout:3241 write 0x2e99400 thru 0x2ea0851 (0x7452 bytes) 2024-09-15T00:09:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:3243 write 0x6567600 thru 0x656dca5 (0x66a6 bytes) 2024-09-15T00:09:55.198 INFO:teuthology.orchestra.run.smithi092.stdout:3246 write 0x2735000 thru 0x273dc36 (0x8c37 bytes) 2024-09-15T00:09:55.199 INFO:teuthology.orchestra.run.smithi092.stdout:3251 write 0x939a800 thru 0x939f116 (0x4917 bytes) 2024-09-15T00:09:55.199 INFO:teuthology.orchestra.run.smithi092.stdout:3252 punch from 0x741e00 to 0x74e800, (0xca00 bytes) 2024-09-15T00:09:55.199 INFO:teuthology.orchestra.run.smithi092.stdout:3253 read 0x6ad4e00 thru 0x6ae06a2 (0xb8a3 bytes) 2024-09-15T00:09:55.201 INFO:teuthology.orchestra.run.smithi092.stdout:3254 read 0x2e84400 thru 0x2e8cbd9 (0x87da bytes) 2024-09-15T00:09:55.209 INFO:teuthology.orchestra.run.smithi092.stdout:3259 trunc from 0xc79f000 to 0x6104c00 2024-09-15T00:09:55.224 INFO:teuthology.orchestra.run.smithi092.stdout:3260 punch from 0x38d2400 to 0x38d8800, (0x6400 bytes) 2024-09-15T00:09:55.226 INFO:teuthology.orchestra.run.smithi092.stdout:3262 punch from 0x384400 to 0x385800, (0x1400 bytes) 2024-09-15T00:09:55.244 INFO:teuthology.orchestra.run.smithi092.stdout:3264 write 0x8dee200 thru 0x8df49b5 (0x67b6 bytes) 2024-09-15T00:09:55.264 INFO:teuthology.orchestra.run.smithi092.stdout:3265 write 0xcfa6c00 thru 0xcfb688e (0xfc8f bytes) 2024-09-15T00:09:55.265 INFO:teuthology.orchestra.run.smithi092.stdout:3269 write 0xbca9c00 thru 0xbcb94d0 (0xf8d1 bytes) 2024-09-15T00:09:55.266 INFO:teuthology.orchestra.run.smithi092.stdout:3270 write 0xab1a200 thru 0xab229bf (0x87c0 bytes) 2024-09-15T00:09:55.266 INFO:teuthology.orchestra.run.smithi092.stdout:3271 read 0x359a00 thru 0x362149 (0x874a bytes) 2024-09-15T00:09:55.267 INFO:teuthology.orchestra.run.smithi092.stdout:3272 punch from 0x26a1a00 to 0x26a4000, (0x2600 bytes) 2024-09-15T00:09:55.267 INFO:teuthology.orchestra.run.smithi092.stdout:3277 write 0xbe29000 thru 0xbe336ea (0xa6eb bytes) 2024-09-15T00:09:55.272 INFO:teuthology.orchestra.run.smithi092.stdout:3278 trunc from 0xcfb6a00 to 0xaa6f600 2024-09-15T00:09:55.277 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:55.275+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:55.287 INFO:teuthology.orchestra.run.smithi092.stdout:3279 read 0x5e24000 thru 0x5e24c81 (0xc82 bytes) 2024-09-15T00:09:55.289 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:55.289 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:55.290 INFO:teuthology.orchestra.run.smithi092.stdout:3281 trunc from 0xaa6f600 to 0xa03ec00 2024-09-15T00:09:55.299 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:55.318 INFO:teuthology.orchestra.run.smithi092.stdout:3285 write 0xec0f600 thru 0xec1b0cc (0xbacd bytes) 2024-09-15T00:09:55.319 INFO:teuthology.orchestra.run.smithi092.stdout:3287 read 0x2a85c00 thru 0x2a8fbda (0x9fdb bytes) 2024-09-15T00:09:55.320 INFO:teuthology.orchestra.run.smithi092.stdout:3289 read 0x32ba400 thru 0x32c64c7 (0xc0c8 bytes) 2024-09-15T00:09:55.320 INFO:teuthology.orchestra.run.smithi092.stdout:3290 write 0x6115800 thru 0x611e74e (0x8f4f bytes) 2024-09-15T00:09:55.331 INFO:teuthology.orchestra.run.smithi092.stdout:3291 trunc from 0xec1b200 to 0x5eee200 2024-09-15T00:09:55.362 INFO:teuthology.orchestra.run.smithi092.stdout:3293 read 0x3134200 thru 0x313fa58 (0xb859 bytes) 2024-09-15T00:09:55.378 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:55.376+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:55.378 INFO:teuthology.orchestra.run.smithi092.stdout:3295 write 0xbe88c00 thru 0xbe8e0d1 (0x54d2 bytes) 2024-09-15T00:09:55.379 INFO:teuthology.orchestra.run.smithi092.stdout:3296 read 0x1226800 thru 0x1235082 (0xe883 bytes) 2024-09-15T00:09:55.380 INFO:teuthology.orchestra.run.smithi092.stdout:3297 read 0x1173600 thru 0x1178203 (0x4c04 bytes) 2024-09-15T00:09:55.380 INFO:teuthology.orchestra.run.smithi092.stdout:3299 write 0x83f200 thru 0x846116 (0x6f17 bytes) 2024-09-15T00:09:55.381 INFO:teuthology.orchestra.run.smithi092.stdout:3300 write 0xacebc00 thru 0xacf352e (0x792f bytes) 2024-09-15T00:09:55.381 INFO:teuthology.orchestra.run.smithi092.stdout:3302 punch from 0x9ba3200 to 0x9baa800, (0x7600 bytes) 2024-09-15T00:09:55.382 INFO:teuthology.orchestra.run.smithi092.stdout:3304 read 0xa7c1a00 thru 0xa7cf69a (0xdc9b bytes) 2024-09-15T00:09:55.384 INFO:teuthology.orchestra.run.smithi092.stdout:3306 punch from 0xaceb000 to 0xacece00, (0x1e00 bytes) 2024-09-15T00:09:55.395 INFO:teuthology.orchestra.run.smithi092.stdout:3307 write 0xe56ce00 thru 0xe56e803 (0x1a04 bytes) 2024-09-15T00:09:55.395 INFO:teuthology.orchestra.run.smithi092.stdout:3308 read 0xbd03800 thru 0xbd0c5e2 (0x8de3 bytes) 2024-09-15T00:09:55.397 INFO:teuthology.orchestra.run.smithi092.stdout:3310 write 0x7f2e800 thru 0x7f367b7 (0x7fb8 bytes) 2024-09-15T00:09:55.397 INFO:teuthology.orchestra.run.smithi092.stdout:3311 read 0x9e00400 thru 0x9e0f69a (0xf29b bytes) 2024-09-15T00:09:55.397 INFO:teuthology.orchestra.run.smithi092.stdout:3312 write 0x7bc0a00 thru 0x7bc6d07 (0x6308 bytes) 2024-09-15T00:09:55.409 INFO:teuthology.orchestra.run.smithi092.stdout:3314 write 0xeacb600 thru 0xeace0ba (0x2abb bytes) 2024-09-15T00:09:55.417 INFO:teuthology.orchestra.run.smithi092.stdout:3316 punch from 0xc7ec200 to 0xc7ec400, (0x200 bytes) 2024-09-15T00:09:55.417 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:55.417 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:55.418 INFO:teuthology.orchestra.run.smithi092.stdout:3317 read 0x54b8200 thru 0x54c4a5c (0xc85d bytes) 2024-09-15T00:09:55.418 INFO:teuthology.orchestra.run.smithi092.stdout:3320 punch from 0x7b9400 to 0x7be200, (0x4e00 bytes) 2024-09-15T00:09:55.418 INFO:teuthology.orchestra.run.smithi092.stdout:3321 punch from 0xe395a00 to 0xe3a0c00, (0xb200 bytes) 2024-09-15T00:09:55.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:55.430 INFO:teuthology.orchestra.run.smithi092.stdout:3324 trunc from 0xeace200 to 0x4b2b000 2024-09-15T00:09:55.451 INFO:teuthology.orchestra.run.smithi092.stdout:3325 punch from 0x697400 to 0x6a3600, (0xc200 bytes) 2024-09-15T00:09:55.456 INFO:teuthology.orchestra.run.smithi092.stdout:3327 read 0x3368a00 thru 0x3374d2d (0xc32e bytes) 2024-09-15T00:09:55.456 INFO:teuthology.orchestra.run.smithi092.stdout:3328 read 0x7cc400 thru 0x7d70a1 (0xaca2 bytes) 2024-09-15T00:09:55.463 INFO:teuthology.orchestra.run.smithi092.stdout:3329 trunc from 0x4b2b000 to 0x9af0400 2024-09-15T00:09:55.478 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:55.476+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:55.485 INFO:teuthology.orchestra.run.smithi092.stdout:3333 write 0xd356600 thru 0xd359f64 (0x3965 bytes) 2024-09-15T00:09:55.490 INFO:teuthology.orchestra.run.smithi092.stdout:3335 write 0x8afd200 thru 0x8b007f4 (0x35f5 bytes) 2024-09-15T00:09:55.503 INFO:teuthology.orchestra.run.smithi092.stdout:3339 trunc from 0xd35a000 to 0x20c7800 2024-09-15T00:09:55.527 INFO:teuthology.orchestra.run.smithi092.stdout:3340 read 0x5e4800 thru 0x5ecb13 (0x8314 bytes) 2024-09-15T00:09:55.537 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:55.537 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:55.546 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:55.550 INFO:teuthology.orchestra.run.smithi092.stdout:3343 write 0xd802400 thru 0xd80b499 (0x909a bytes) 2024-09-15T00:09:55.552 INFO:teuthology.orchestra.run.smithi092.stdout:3344 punch from 0x1192200 to 0x119be00, (0x9c00 bytes) 2024-09-15T00:09:55.553 INFO:teuthology.orchestra.run.smithi092.stdout:3345 read 0x148f800 thru 0x14954d4 (0x5cd5 bytes) 2024-09-15T00:09:55.553 INFO:teuthology.orchestra.run.smithi092.stdout:3348 write 0x839fa00 thru 0x83a2f81 (0x3582 bytes) 2024-09-15T00:09:55.566 INFO:teuthology.orchestra.run.smithi092.stdout:3349 trunc from 0xd80b600 to 0x4983600 2024-09-15T00:09:55.579 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:55.578+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:55.584 INFO:teuthology.orchestra.run.smithi092.stdout:3351 punch from 0x327800 to 0x335400, (0xdc00 bytes) 2024-09-15T00:09:55.585 INFO:teuthology.orchestra.run.smithi092.stdout:3354 read 0x10a8600 thru 0x10a86f5 (0xf6 bytes) 2024-09-15T00:09:55.586 INFO:teuthology.orchestra.run.smithi092.stdout:3358 read 0x3280600 thru 0x32875ac (0x6fad bytes) 2024-09-15T00:09:55.600 INFO:teuthology.orchestra.run.smithi092.stdout:3359 write 0x82d2400 thru 0x82d3a89 (0x168a bytes) 2024-09-15T00:09:55.612 INFO:teuthology.orchestra.run.smithi092.stdout:3360 trunc from 0x82d3c00 to 0xd98fa00 2024-09-15T00:09:55.623 INFO:teuthology.orchestra.run.smithi092.stdout:3361 write 0x355b000 thru 0x3564bbf (0x9bc0 bytes) 2024-09-15T00:09:55.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:09:55.629 INFO:teuthology.orchestra.run.smithi092.stdout:3363 read 0xa42600 thru 0xa4a3a6 (0x7da7 bytes) 2024-09-15T00:09:55.630 INFO:teuthology.orchestra.run.smithi092.stdout:3364 write 0xb08a400 thru 0xb08e3ea (0x3feb bytes) 2024-09-15T00:09:55.641 INFO:teuthology.orchestra.run.smithi092.stdout:3365 trunc from 0xd98fa00 to 0x45be200 2024-09-15T00:09:55.657 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:55.657 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:55.664 INFO:teuthology.orchestra.run.smithi092.stdout:3366 trunc from 0x45be200 to 0x327400 2024-09-15T00:09:55.666 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:55.680 INFO:teuthology.orchestra.run.smithi092.stdout:3368 punch from 0x1ae200 to 0x1ae800, (0x600 bytes) 2024-09-15T00:09:55.680 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:55.679+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:55.709 INFO:teuthology.orchestra.run.smithi092.stdout:3371 write 0xce63000 thru 0xce71b75 (0xeb76 bytes) 2024-09-15T00:09:55.710 INFO:teuthology.orchestra.run.smithi092.stdout:3372 punch from 0x73f0600 to 0x73fde00, (0xd800 bytes) 2024-09-15T00:09:55.710 INFO:teuthology.orchestra.run.smithi092.stdout:3374 read 0xc699800 thru 0xc6a5b51 (0xc352 bytes) 2024-09-15T00:09:55.710 INFO:teuthology.orchestra.run.smithi092.stdout:3375 punch from 0xc3ace00 to 0xc3b8a00, (0xbc00 bytes) 2024-09-15T00:09:55.712 INFO:teuthology.orchestra.run.smithi092.stdout:3376 trunc from 0xce71c00 to 0xe846e00 2024-09-15T00:09:55.724 INFO:teuthology.orchestra.run.smithi092.stdout:3378 trunc from 0xe846e00 to 0xcda7600 2024-09-15T00:09:55.736 INFO:teuthology.orchestra.run.smithi092.stdout:3379 read 0xb66c200 thru 0xb67704b (0xae4c bytes) 2024-09-15T00:09:55.737 INFO:teuthology.orchestra.run.smithi092.stdout:3380 write 0xa6b4200 thru 0xa6bff8c (0xbd8d bytes) 2024-09-15T00:09:55.738 INFO:teuthology.orchestra.run.smithi092.stdout:3384 punch from 0x5d13600 to 0x5d1c000, (0x8a00 bytes) 2024-09-15T00:09:55.738 INFO:teuthology.orchestra.run.smithi092.stdout:3386 read 0x4d1c00 thru 0x4dccda (0xb0db bytes) 2024-09-15T00:09:55.738 INFO:teuthology.orchestra.run.smithi092.stdout:3389 write 0xb848000 thru 0xb850a04 (0x8a05 bytes) 2024-09-15T00:09:55.739 INFO:teuthology.orchestra.run.smithi092.stdout:3390 write 0x2579000 thru 0x2588022 (0xf023 bytes) 2024-09-15T00:09:55.739 INFO:teuthology.orchestra.run.smithi092.stdout:3391 read 0x951000 thru 0x95b4d2 (0xa4d3 bytes) 2024-09-15T00:09:55.740 INFO:teuthology.orchestra.run.smithi092.stdout:3395 write 0x8737a00 thru 0x8737f15 (0x516 bytes) 2024-09-15T00:09:55.740 INFO:teuthology.orchestra.run.smithi092.stdout:3396 write 0x4a6f400 thru 0x4a74a8e (0x568f bytes) 2024-09-15T00:09:55.741 INFO:teuthology.orchestra.run.smithi092.stdout:3397 punch from 0x97cc400 to 0x97da400, (0xe000 bytes) 2024-09-15T00:09:55.742 INFO:teuthology.orchestra.run.smithi092.stdout:3398 punch from 0x8db8800 to 0x8dc6e00, (0xe600 bytes) 2024-09-15T00:09:55.742 INFO:teuthology.orchestra.run.smithi092.stdout:3400 read 0x5900a00 thru 0x590efcc (0xe5cd bytes) 2024-09-15T00:09:55.742 INFO:teuthology.orchestra.run.smithi092.stdout:3402 write 0xbb37200 thru 0xbb3960e (0x240f bytes) 2024-09-15T00:09:55.744 INFO:teuthology.orchestra.run.smithi092.stdout:3403 write 0xcc85600 thru 0xcc8da52 (0x8453 bytes) 2024-09-15T00:09:55.744 INFO:teuthology.orchestra.run.smithi092.stdout:3408 write 0x94f5600 thru 0x94f9d32 (0x4733 bytes) 2024-09-15T00:09:55.745 INFO:teuthology.orchestra.run.smithi092.stdout:3410 read 0x4e22a00 thru 0x4e300c7 (0xd6c8 bytes) 2024-09-15T00:09:55.745 INFO:teuthology.orchestra.run.smithi092.stdout:3412 punch from 0xa670600 to 0xa675400, (0x4e00 bytes) 2024-09-15T00:09:55.746 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:09:55.759 INFO:teuthology.orchestra.run.smithi092.stdout:3413 trunc from 0xcda7600 to 0x14b4400 2024-09-15T00:09:55.766 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:55.766 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:55.782 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:55.780+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:55.784 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:55.785 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:55.791 INFO:teuthology.orchestra.run.smithi092.stdout:3414 punch from 0x39b400 to 0x3a7e00, (0xca00 bytes) 2024-09-15T00:09:55.791 INFO:teuthology.orchestra.run.smithi092.stdout:3415 read 0x7b5000 thru 0x7b8e78 (0x3e79 bytes) 2024-09-15T00:09:55.801 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:55.801 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:55.803 INFO:teuthology.orchestra.run.smithi092.stdout:3417 trunc from 0x14b4400 to 0xad75600 2024-09-15T00:09:55.819 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:55.819 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:55.821 INFO:teuthology.orchestra.run.smithi092.stdout:3418 trunc from 0xad75600 to 0x240ac00 2024-09-15T00:09:55.822 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:55.822 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:55.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:55.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:55.837 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:55.840 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:09:55.840 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:09:55.852 INFO:teuthology.orchestra.run.smithi092.stdout:3423 write 0x596d800 thru 0x597a163 (0xc964 bytes) 2024-09-15T00:09:55.869 INFO:teuthology.orchestra.run.smithi092.stdout:3426 write 0x63a3c00 thru 0x63acaf6 (0x8ef7 bytes) 2024-09-15T00:09:55.873 INFO:teuthology.orchestra.run.smithi092.stdout:3427 trunc from 0x63acc00 to 0x3e9ce00 2024-09-15T00:09:55.883 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:55.881+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:55.902 INFO:teuthology.orchestra.run.smithi092.stdout:3429 write 0x902f800 thru 0x903c270 (0xca71 bytes) 2024-09-15T00:09:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:3430 punch from 0x78e5e00 to 0x78e9600, (0x3800 bytes) 2024-09-15T00:09:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:3431 read 0x369c000 thru 0x36aba22 (0xfa23 bytes) 2024-09-15T00:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout:3433 write 0xb03b200 thru 0xb045624 (0xa425 bytes) 2024-09-15T00:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout:3435 punch from 0x6b4d200 to 0x6b53000, (0x5e00 bytes) 2024-09-15T00:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout:3436 read 0x724f000 thru 0x7253c63 (0x4c64 bytes) 2024-09-15T00:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout:3438 punch from 0x13200 to 0x19800, (0x6600 bytes) 2024-09-15T00:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout:3439 read 0x42c1c00 thru 0x42cabfe (0x8fff bytes) 2024-09-15T00:09:55.927 INFO:teuthology.orchestra.run.smithi092.stdout:3440 trunc from 0xb045800 to 0x5446e00 2024-09-15T00:09:55.951 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:55.951 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:09:55.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:09:55.951 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.923394177999999, 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:09:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:09:55.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:09:55.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:55.956 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.923372260000001, 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:55.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:09:55.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.923350987999999, 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.961 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.921609728, 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:09:55.965 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:09:55.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:09:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.921587821999999, 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:09:55.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.921565730000001, 2024-09-15T00:09:55.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:55.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.921533991, 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:09:55.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:09:55.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.921235788000001, 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:09:55.983 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:09:55.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:09:55.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.921202083000001, 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.988 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.916900009999999, 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:09:55.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:55.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:09:55.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:09:55.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:55.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:09:55.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:09:55.996 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.996 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:55.996 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.916575139999999, 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:55.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:09:55.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:56.001 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.916164734999999, 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:56.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:09:56.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.916109832, 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:56.006 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:09:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:56.010 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.916077797, 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:56.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.014 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.914599967000001, 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:09:56.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:09:56.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:09:56.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.914578251999998, 2024-09-15T00:09:56.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:09:56.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.023 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.914556773000001, 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:56.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:09:56.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:56.028 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.912543669000002, 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:09:56.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.912524028, 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:09:56.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:56.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:09:56.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.91247748, 2024-09-15T00:09:56.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:56.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:09:56.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:09:56.041 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.042 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.042 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.042 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:09:56.042 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:09:56.042 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:56.042 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:09:56.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:56.045 INFO:teuthology.orchestra.run.smithi027.stdout: marking collection for removal 2024-09-15T00:09:56.046 INFO:teuthology.orchestra.run.smithi027.stdout:setting '_remove' omap key 2024-09-15T00:09:56.046 INFO:teuthology.orchestra.run.smithi027.stdout:finish_remove_pgs 3.c_head removing 3.c 2024-09-15T00:09:56.046 INFO:teuthology.orchestra.run.smithi027.stdout:Remove successful 2024-09-15T00:09:56.046 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:56.042+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3442 write 0xbcbcc00 thru 0xbcc9210 (0xc611 bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3445 write 0xedefa00 thru 0xedf6fe1 (0x75e2 bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3446 read 0x3df3400 thru 0x3df9998 (0x6599 bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3450 punch from 0xe34fe00 to 0xe35de00, (0xe000 bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3453 read 0xc1a6000 thru 0xc1adf14 (0x7f15 bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3454 read 0x4339e00 thru 0x434423e (0xa43f bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3456 punch from 0xe5cec00 to 0xe5d9600, (0xaa00 bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3457 write 0x9443e00 thru 0x944681f (0x2a20 bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3458 trunc from 0xedf7000 to 0x464b200 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3460 write 0x4da7c00 thru 0x4daee8e (0x728f bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3462 read 0x21d600 thru 0x21fd4b (0x274c bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3463 punch from 0x2963a00 to 0x296e600, (0xac00 bytes) 2024-09-15T00:09:56.047 INFO:teuthology.orchestra.run.smithi092.stdout:3465 trunc from 0x4daf000 to 0xb5c3600 2024-09-15T00:09:56.054 INFO:teuthology.orchestra.run.smithi092.stdout:3466 punch from 0xf06000 to 0xf0be00, (0x5e00 bytes) 2024-09-15T00:09:56.055 INFO:teuthology.orchestra.run.smithi092.stdout:3468 punch from 0x5cd7600 to 0x5cdf000, (0x7a00 bytes) 2024-09-15T00:09:56.055 INFO:teuthology.orchestra.run.smithi092.stdout:3470 read 0x6782400 thru 0x6782f10 (0xb11 bytes) 2024-09-15T00:09:56.059 INFO:teuthology.orchestra.run.smithi092.stdout:3473 read 0x360a200 thru 0x3619cfe (0xfaff bytes) 2024-09-15T00:09:56.060 INFO:teuthology.orchestra.run.smithi092.stdout:3476 read 0xd04600 thru 0xd099f7 (0x53f8 bytes) 2024-09-15T00:09:56.060 INFO:teuthology.orchestra.run.smithi092.stdout:3479 read 0x74fdc00 thru 0x750d47e (0xf87f bytes) 2024-09-15T00:09:56.061 INFO:teuthology.orchestra.run.smithi092.stdout:3480 punch from 0x9ea4c00 to 0x9eac400, (0x7800 bytes) 2024-09-15T00:09:56.061 INFO:teuthology.orchestra.run.smithi092.stdout:3484 punch from 0x41d000 to 0x420a00, (0x3a00 bytes) 2024-09-15T00:09:56.061 INFO:teuthology.orchestra.run.smithi092.stdout:3486 punch from 0xabbde00 to 0xabcb200, (0xd400 bytes) 2024-09-15T00:09:56.061 INFO:teuthology.orchestra.run.smithi092.stdout:3489 read 0x94ac200 thru 0x94b8d63 (0xcb64 bytes) 2024-09-15T00:09:56.061 INFO:teuthology.orchestra.run.smithi092.stdout:3490 write 0xa32ca00 thru 0xa33757f (0xab80 bytes) 2024-09-15T00:09:56.075 INFO:teuthology.orchestra.run.smithi092.stdout:3491 trunc from 0xb5c3600 to 0x8a1600 2024-09-15T00:09:56.109 INFO:teuthology.orchestra.run.smithi092.stdout:3492 write 0x16f7c00 thru 0x16ffc8e (0x808f bytes) 2024-09-15T00:09:56.122 INFO:teuthology.orchestra.run.smithi092.stdout:3494 trunc from 0x16ffe00 to 0xb4a6600 2024-09-15T00:09:56.137 INFO:teuthology.orchestra.run.smithi092.stdout:3495 write 0x4d84400 thru 0x4d9009f (0xbca0 bytes) 2024-09-15T00:09:56.139 INFO:teuthology.orchestra.run.smithi092.stdout:3498 trunc from 0xb4a6600 to 0x7c81600 2024-09-15T00:09:56.144 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:56.143+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:56.155 INFO:teuthology.orchestra.run.smithi092.stdout:3502 punch from 0x6df4200 to 0x6e00000, (0xbe00 bytes) 2024-09-15T00:09:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:56.160 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:56.163 INFO:teuthology.orchestra.run.smithi092.stdout:3503 trunc from 0x7c81600 to 0x896b800 2024-09-15T00:09:56.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:56.182 INFO:teuthology.orchestra.run.smithi092.stdout:3504 trunc from 0x896b800 to 0x1dca400 2024-09-15T00:09:56.197 INFO:teuthology.orchestra.run.smithi092.stdout:3505 punch from 0x24b400 to 0x259e00, (0xea00 bytes) 2024-09-15T00:09:56.205 INFO:teuthology.orchestra.run.smithi092.stdout:3507 punch from 0x11c4600 to 0x11d3400, (0xee00 bytes) 2024-09-15T00:09:56.205 INFO:teuthology.orchestra.run.smithi092.stdout:3508 read 0xa15200 thru 0xa17020 (0x1e21 bytes) 2024-09-15T00:09:56.225 INFO:teuthology.orchestra.run.smithi092.stdout:3509 write 0xa0c3200 thru 0xa0c6e40 (0x3c41 bytes) 2024-09-15T00:09:56.233 INFO:teuthology.orchestra.run.smithi092.stdout:3510 read 0x55be00 thru 0x560d8b (0x4f8c bytes) 2024-09-15T00:09:56.233 INFO:teuthology.orchestra.run.smithi092.stdout:3511 read 0x1b61c00 thru 0x1b6dfed (0xc3ee bytes) 2024-09-15T00:09:56.234 INFO:teuthology.orchestra.run.smithi092.stdout:3512 read 0x886600 thru 0x88fc5c (0x965d bytes) 2024-09-15T00:09:56.234 INFO:teuthology.orchestra.run.smithi092.stdout:3513 read 0x8eb4600 thru 0x8ebcaf2 (0x84f3 bytes) 2024-09-15T00:09:56.234 INFO:teuthology.orchestra.run.smithi092.stdout:3515 read 0x80bae00 thru 0x80ca5fd (0xf7fe bytes) 2024-09-15T00:09:56.235 INFO:teuthology.orchestra.run.smithi092.stdout:3520 write 0x1a23e00 thru 0x1a2ba11 (0x7c12 bytes) 2024-09-15T00:09:56.235 INFO:teuthology.orchestra.run.smithi092.stdout:3521 read 0x40cdc00 thru 0x40d186e (0x3c6f bytes) 2024-09-15T00:09:56.236 INFO:teuthology.orchestra.run.smithi092.stdout:3522 read 0x402ba00 thru 0x40331c1 (0x77c2 bytes) 2024-09-15T00:09:56.236 INFO:teuthology.orchestra.run.smithi092.stdout:3523 write 0x1f8a600 thru 0x1f8c2f7 (0x1cf8 bytes) 2024-09-15T00:09:56.237 INFO:teuthology.orchestra.run.smithi092.stdout:3524 read 0x8cf8a00 thru 0x8cff70f (0x6d10 bytes) 2024-09-15T00:09:56.249 INFO:teuthology.orchestra.run.smithi092.stdout:3525 trunc from 0xa0c7000 to 0x5cec00 2024-09-15T00:09:56.253 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:56.252+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:56.268 INFO:teuthology.orchestra.run.smithi092.stdout:3527 read 0x1a1e00 thru 0x1a61e2 (0x43e3 bytes) 2024-09-15T00:09:56.273 INFO:teuthology.orchestra.run.smithi092.stdout:3528 read 0x388000 thru 0x38aafa (0x2afb bytes) 2024-09-15T00:09:56.273 INFO:teuthology.orchestra.run.smithi092.stdout:3531 read 0x2a6800 thru 0x2aa596 (0x3d97 bytes) 2024-09-15T00:09:56.298 INFO:teuthology.orchestra.run.smithi092.stdout:3534 write 0xc9a7200 thru 0xc9b3a8e (0xc88f bytes) 2024-09-15T00:09:56.299 INFO:teuthology.orchestra.run.smithi092.stdout:3539 write 0xa9e8000 thru 0xa9eeee5 (0x6ee6 bytes) 2024-09-15T00:09:56.299 INFO:teuthology.orchestra.run.smithi092.stdout:3542 punch from 0xa8a0000 to 0xa8afc00, (0xfc00 bytes) 2024-09-15T00:09:56.300 INFO:teuthology.orchestra.run.smithi092.stdout:3547 write 0x922f800 thru 0x9234ee3 (0x56e4 bytes) 2024-09-15T00:09:56.300 INFO:teuthology.orchestra.run.smithi092.stdout:3549 punch from 0x9f74200 to 0x9f7b400, (0x7200 bytes) 2024-09-15T00:09:56.300 INFO:teuthology.orchestra.run.smithi092.stdout:3550 read 0x25f5400 thru 0x25f74c9 (0x20ca bytes) 2024-09-15T00:09:56.313 INFO:teuthology.orchestra.run.smithi092.stdout:3551 trunc from 0xc9b3c00 to 0x2b3a600 2024-09-15T00:09:56.322 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:56.322 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:56.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:56.346 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:56.345+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:56.354 INFO:teuthology.orchestra.run.smithi092.stdout:3553 write 0xcaf8000 thru 0xcaf9656 (0x1657 bytes) 2024-09-15T00:09:56.355 INFO:teuthology.orchestra.run.smithi092.stdout:3554 clone 26 order 20 su 8192 sc 2 2024-09-15T00:09:56.447 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:56.446+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:56.477 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:56.477 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4070 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124374680999999, 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02702953, 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4070 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350999+0000", 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351003+0000", 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351007+0000", 2024-09-15T00:09:56.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6820000000000001e-06 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351010+0000", 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7269999999999999e-06 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377929+0000", 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026919887999999999 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377969+0000", 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9909e-05 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378028+0000", 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8578000000000001e-05 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4071 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:56.482 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124359800999997, 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027136008, 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4071 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351014+0000", 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351015+0000", 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7039999999999999e-06 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351020+0000", 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351023+0000", 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378053+0000", 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027029610999999999 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378094+0000", 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1826999999999998e-05 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378149+0000", 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4815000000000003e-05 2024-09-15T00:09:56.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4072 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124335121999998, 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027223404999999999, 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.487 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4072 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351040+0000", 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351046+0000", 2024-09-15T00:09:56.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351051+0000", 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378171+0000", 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027120590999999999 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378209+0000", 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7308999999999998e-05 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378261+0000", 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2753999999999997e-05 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4074 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.491 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124316227000001, 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027334509999999999, 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4074 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:09:56.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9200000000000002e-07 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351059+0000", 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.012e-06 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351065+0000", 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2129999999999996e-06 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351070+0000", 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8470000000000003e-06 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378285+0000", 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027214449000000002 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378334+0000", 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378391+0000", 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7046000000000002e-05 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4075 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124297356, 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027546923000000001, 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4075 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351078+0000", 2024-09-15T00:09:56.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.905e-06 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351085+0000", 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351090+0000", 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378423+0000", 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027333051000000001 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378523+0000", 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9959999999999998e-05 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378623+0000", 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9962000000000003e-05 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4077 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124274671999999, 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027660619000000001, 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.501 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4077 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351099+0000", 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351101+0000", 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351107+0000", 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7980000000000002e-06 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351111+0000", 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378657+0000", 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027545463999999999 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378702+0000", 2024-09-15T00:09:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5809999999999997e-05 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378759+0000", 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4078 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:56.505 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124246671999998, 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027808405000000001, 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4078 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351127+0000", 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351129+0000", 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351135+0000", 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3820000000000001e-06 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351141+0000", 2024-09-15T00:09:56.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2259999999999996e-06 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378785+0000", 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027644592999999999 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378855+0000", 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0276000000000003e-05 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378935+0000", 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9361000000000001e-05 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4079 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124225035000002, 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027913778, 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4079 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:56.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351150+0000", 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9010000000000001e-06 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351157+0000", 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351162+0000", 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378958+0000", 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027795786999999999 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379004+0000", 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6124000000000001e-05 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379062+0000", 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7630999999999998e-05 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4081 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124201354, 2024-09-15T00:09:56.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016658, 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4081 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351174+0000", 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351181+0000", 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9070000000000001e-06 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351186+0000", 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.651e-06 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379085+0000", 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027898757999999999 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379130+0000", 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5126000000000001e-05 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379188+0000", 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8548999999999998e-05 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.518 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4082 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124181299999996, 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028118457999999999, 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4082 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351194+0000", 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7519999999999999e-06 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351200+0000", 2024-09-15T00:09:56.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.801e-06 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351205+0000", 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379213+0000", 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028007539000000001 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379254+0000", 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1817999999999998e-05 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379310+0000", 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5850000000000002e-05 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4083 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.523 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124159579999997, 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028218838999999999, 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4083 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351214+0000", 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:56.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351216+0000", 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351222+0000", 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351226+0000", 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379333+0000", 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028106740000000002 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379376+0000", 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3581000000000002e-05 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379432+0000", 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5868000000000001e-05 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4085 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124131075999998, 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311823, 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.528 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4085 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0699999999999998e-07 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351244+0000", 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.782e-06 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351248+0000", 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6289999999999999e-06 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351250+0000", 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379457+0000", 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028206479 2024-09-15T00:09:56.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379499+0000", 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1637000000000001e-05 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379554+0000", 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5248999999999999e-05 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4086 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124119235000002, 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02842368, 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4086 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:09:56.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351259+0000", 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9520000000000002e-06 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351262+0000", 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351265+0000", 2024-09-15T00:09:56.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379576+0000", 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311228000000001 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379619+0000", 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2595000000000001e-05 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379678+0000", 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4087 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124104879999997, 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523903999999999, 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.537 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4087 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351269+0000", 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351270+0000", 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1090000000000001e-06 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351273+0000", 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9189999999999999e-06 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351275+0000", 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379701+0000", 2024-09-15T00:09:56.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028425813000000001 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379742+0000", 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0769000000000002e-05 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379792+0000", 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4088 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124088745999998, 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028621284, 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4088 2024-09-15T00:09:56.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351285+0000", 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351286+0000", 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.186e-06 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351289+0000", 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351292+0000", 2024-09-15T00:09:56.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:09:56.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379815+0000", 2024-09-15T00:09:56.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523211 2024-09-15T00:09:56.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379857+0000", 2024-09-15T00:09:56.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1807999999999997e-05 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379906+0000", 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8594999999999998e-05 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4090 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124077661000001, 2024-09-15T00:09:56.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028748587999999999, 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4090 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:56.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351296+0000", 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351297+0000", 2024-09-15T00:09:56.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1060000000000001e-06 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351300+0000", 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9110000000000002e-06 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351302+0000", 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379947+0000", 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028645177000000001 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379991+0000", 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3680999999999997e-05 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380044+0000", 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2957000000000001e-05 2024-09-15T00:09:56.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4091 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124068106000003, 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028864198000000001, 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4091 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351306+0000", 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351309+0000", 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-15T00:09:56.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351312+0000", 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380069+0000", 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028757933999999999 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380113+0000", 2024-09-15T00:09:56.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3810000000000002e-05 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380169+0000", 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5887999999999998e-05 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4092 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124058748000003, 2024-09-15T00:09:56.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028973819000000001, 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4092 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:56.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351315+0000", 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9999999999999999e-07 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351316+0000", 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.95e-07 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351318+0000", 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8629999999999999e-06 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351321+0000", 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380195+0000", 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028873868 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380238+0000", 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2883999999999998e-05 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380288+0000", 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0096000000000003e-05 2024-09-15T00:09:56.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4094 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124049128999999, 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02907964, 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4094 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351325+0000", 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.054e-06 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351328+0000", 2024-09-15T00:09:56.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351331+0000", 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4679999999999999e-06 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380312+0000", 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028981587999999999 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380355+0000", 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2688000000000002e-05 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380404+0000", 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8767999999999998e-05 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.567 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4095 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.124039742000001, 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029185068000000002, 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4095 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351334+0000", 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.04e-07 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351335+0000", 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0300000000000001e-06 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351339+0000", 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:56.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351344+0000", 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0560000000000004e-06 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380428+0000", 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02908444 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380470+0000", 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1338000000000003e-05 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:56.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:56.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380518+0000", 2024-09-15T00:09:56.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8724999999999999e-05 2024-09-15T00:09:56.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:56.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:56.572 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:56.574 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.c.0 2024-09-15T00:09:56.574 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:56.572+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:56.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:56.675 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:56.673+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:56.718 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:56.718 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:56.718 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:56.718 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:56.727 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:56.775 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:56.774+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:56.876 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:56.874+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:56.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:56.976 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:56.975+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:57.023 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:09:57.025 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:57.025 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:57.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:57.025 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3427 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735424340000002, 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027724631, 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3427 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287582+0000", 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2599999999999998e-07 2024-09-15T00:09:57.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287584+0000", 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8759999999999999e-06 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287589+0000", 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287594+0000", 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.357e-06 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315217+0000", 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027622785 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315257+0000", 2024-09-15T00:09:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0228000000000002e-05 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315306+0000", 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9228999999999999e-05 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.030 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3428 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735406335, 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027928562000000001, 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3428 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287600+0000", 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:09:57.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287602+0000", 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287608+0000", 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287613+0000", 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8740000000000003e-06 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315346+0000", 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027733099000000001 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315431+0000", 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4350000000000004e-05 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315528+0000", 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7374000000000007e-05 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3429 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735387025000001, 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028028630999999998, 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.035 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3429 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287621+0000", 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287627+0000", 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8440000000000003e-06 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287631+0000", 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5009999999999998e-06 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315556+0000", 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027925241999999999 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315595+0000", 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8512e-05 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315647+0000", 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2318000000000003e-05 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.039 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3430 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735368950999998, 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028122114, 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3430 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:09:57.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287644+0000", 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4219999999999997e-06 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287650+0000", 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0920000000000003e-06 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287654+0000", 2024-09-15T00:09:57.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4279999999999998e-06 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315671+0000", 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016592 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315709+0000", 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8231e-05 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315759+0000", 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9926999999999998e-05 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3431 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735345410999997, 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02831842, 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.044 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3431 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287661+0000", 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287663+0000", 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9570000000000001e-06 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287668+0000", 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287673+0000", 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315799+0000", 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028125741999999999 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315891+0000", 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2701000000000006e-05 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315979+0000", 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7287000000000006e-05 2024-09-15T00:09:57.048 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3432 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735327026999997, 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028430624000000002, 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3432 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:57.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287681+0000", 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8530000000000001e-06 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287686+0000", 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4659999999999998e-06 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287691+0000", 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2230000000000001e-06 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316006+0000", 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028315344999999999 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316054+0000", 2024-09-15T00:09:57.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8004999999999999e-05 2024-09-15T00:09:57.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316109+0000", 2024-09-15T00:09:57.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5374999999999998e-05 2024-09-15T00:09:57.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.053 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3433 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735309270999998, 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028528593000000001, 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.054 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3433 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287697+0000", 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287699+0000", 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8509999999999999e-06 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287704+0000", 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:09:57.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287710+0000", 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316124+0000", 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028414466999999999 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316171+0000", 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6783999999999998e-05 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316225+0000", 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3891999999999998e-05 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3434 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.058 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735289885999997, 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028632801999999999, 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3434 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-07 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287718+0000", 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287724+0000", 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8390000000000003e-06 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287729+0000", 2024-09-15T00:09:57.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8389999999999997e-06 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316250+0000", 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028521135 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316295+0000", 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4540999999999999e-05 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316349+0000", 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4083999999999999e-05 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3435 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735259210000002, 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028723243999999998, 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.063 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3435 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287747+0000", 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4900000000000002e-07 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287748+0000", 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4190000000000001e-06 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287752+0000", 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7900000000000001e-06 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287756+0000", 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1609999999999996e-06 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316375+0000", 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028618932 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316417+0000", 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316470+0000", 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3436 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:57.067 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735244162000001, 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028827854, 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3436 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287766+0000", 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287771+0000", 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5140000000000001e-06 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287776+0000", 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316494+0000", 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028718144000000001 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316538+0000", 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4129999999999999e-05 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316589+0000", 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1548999999999997e-05 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3437 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735226269999998, 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028972188999999999, 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3437 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287780+0000", 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287782+0000", 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8419999999999999e-06 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287787+0000", 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3059999999999997e-06 2024-09-15T00:09:57.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287791+0000", 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9920000000000001e-06 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316615+0000", 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028824487999999999 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316668+0000", 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2883999999999997e-05 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316752+0000", 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3308999999999998e-05 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3438 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.076 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735209228000002, 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029124331999999999, 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3438 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287797+0000", 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287799+0000", 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287804+0000", 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7069999999999997e-06 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287808+0000", 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0720000000000001e-06 2024-09-15T00:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316799+0000", 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028990241999999999 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316845+0000", 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5837000000000002e-05 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316921+0000", 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6255999999999996e-05 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3439 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735191688999997, 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029290665, 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3439 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:57.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287816+0000", 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6980000000000001e-06 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287824+0000", 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2500000000000006e-06 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287829+0000", 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316967+0000", 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029138319999999999 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317047+0000", 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0081999999999997e-05 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317105+0000", 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7497999999999997e-05 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.085 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3440 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735170629999999, 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029395451999999999, 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3440 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:09:57.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287837+0000", 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287843+0000", 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6799999999999998e-06 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287848+0000", 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317131+0000", 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029283337999999999 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317176+0000", 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5176999999999998e-05 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317231+0000", 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4120999999999999e-05 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3441 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735152565999996, 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029496321999999998, 2024-09-15T00:09:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3441 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287855+0000", 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7099999999999999e-06 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287861+0000", 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287866+0000", 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317252+0000", 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029386051999999999 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317297+0000", 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5164999999999999e-05 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317349+0000", 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2534000000000002e-05 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3442 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735133300000001, 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029594697999999999, 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3442 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287873+0000", 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287879+0000", 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287885+0000", 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.755e-06 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287890+0000", 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317375+0000", 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029485457 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317416+0000", 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1217999999999997e-05 2024-09-15T00:09:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317467+0000", 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0819999999999998e-05 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3443 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735109960999999, 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029684676, 2024-09-15T00:09:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3443 2024-09-15T00:09:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:09:57.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287898+0000", 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287904+0000", 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8860000000000001e-06 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287908+0000", 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5650000000000003e-06 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317488+0000", 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029579437 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317530+0000", 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1900999999999997e-05 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317580+0000", 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0603000000000003e-05 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3444 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735084743000002, 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029772823, 2024-09-15T00:09:57.104 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3444 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287931+0000", 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0008e-05 2024-09-15T00:09:57.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287938+0000", 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287943+0000", 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317603+0000", 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029659913999999999 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317646+0000", 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2367999999999998e-05 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317694+0000", 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8201999999999997e-05 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3445 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735055404999997, 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029863565000000002, 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3445 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287951+0000", 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287953+0000", 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.765e-06 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287958+0000", 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999996e-06 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287963+0000", 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.634e-06 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317714+0000", 2024-09-15T00:09:57.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029751353000000001 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317755+0000", 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0342000000000001e-05 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317814+0000", 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9188999999999999e-05 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3446 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.735037106999997, 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029967884, 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:09:57.114 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3446 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287971+0000", 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287977+0000", 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4330000000000003e-06 2024-09-15T00:09:57.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287981+0000", 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3989999999999997e-06 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317837+0000", 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029855942 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317878+0000", 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467000000000001e-05 2024-09-15T00:09:57.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317936+0000", 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8100999999999998e-05 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.118 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:57.120 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:57.118+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:57.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:57.220 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:57.219+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:57.266 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:57.266 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:57.266 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:09:57.266 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:57.275 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:57.321 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:57.320+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:57.422 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:57.421+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:57.425 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:57.425 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:09:57.425 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:09:57.425 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:09:57.426 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:57.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:57.465 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-15T00:09:57.523 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:57.522+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:57.562 INFO:teuthology.orchestra.run.smithi092.stdout:3556 write 0x11a5c00 thru 0x11acfb2 (0x73b3 bytes) 2024-09-15T00:09:57.564 INFO:teuthology.orchestra.run.smithi092.stdout:3558 punch from 0x630fc00 to 0x6311e00, (0x2200 bytes) 2024-09-15T00:09:57.593 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:09:57.593 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:09:57.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:09:57.593 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:09:57.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.593 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1142 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.593 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.707146614000003, 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045190363999999997, 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1142 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882755+0000", 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882756+0000", 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882773+0000", 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6872999999999999e-05 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882780+0000", 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4240000000000004e-06 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882787+0000", 2024-09-15T00:09:57.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8510000000000001e-06 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927799+0000", 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045011974000000003 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927843+0000", 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4270999999999997e-05 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927946+0000", 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102581 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.707097922000003, 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.598 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:09:57.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.602 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.707059291999997, 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.607 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.707033355, 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:09:57.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:09:57.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:09:57.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:09:57.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:09:57.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.613 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.614 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.614 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:57.614 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.707017286999999, 2024-09-15T00:09:57.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:09:57.615 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.615 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.615 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.616 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:09:57.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.616 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:57.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:09:57.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:09:57.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:09:57.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:09:57.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:09:57.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:09:57.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:09:57.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:09:57.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:09:57.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:09:57.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:09:57.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:09:57.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:09:57.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:09:57.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:09:57.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.706876192000003, 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:09:57.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:09:57.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.706104437, 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:09:57.629 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:09:57.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.634 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.634 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.634 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:57.634 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.706019406000003, 2024-09-15T00:09:57.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:09:57.635 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.635 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:57.639 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.705984389999998, 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:57.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:09:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.705958787, 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.644 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.645 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.645 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.645 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.645 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:09:57.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.645 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:57.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:09:57.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:09:57.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.649 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.705784520999998, 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:09:57.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:09:57.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:09:57.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:09:57.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.705553381999998, 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.660 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1157 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.660 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:57.660 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.704771944000001, 2024-09-15T00:09:57.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045184281, 2024-09-15T00:09:57.660 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.661 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.661 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.661 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1157 2024-09-15T00:09:57.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.662 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:57.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:57.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885130+0000", 2024-09-15T00:09:57.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:09:57.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885132+0000", 2024-09-15T00:09:57.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-15T00:09:57.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885139+0000", 2024-09-15T00:09:57.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0350000000000004e-06 2024-09-15T00:09:57.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885153+0000", 2024-09-15T00:09:57.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3515e-05 2024-09-15T00:09:57.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930177+0000", 2024-09-15T00:09:57.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045024495999999997 2024-09-15T00:09:57.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930249+0000", 2024-09-15T00:09:57.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2072999999999999e-05 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930314+0000", 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4826999999999994e-05 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.668 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.704741437999999, 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:09:57.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.704661698000002, 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.673 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:09:57.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.704456673000003, 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:09:57.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.704245139999998, 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.682 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:57.686 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.701689133000002, 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:57.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:09:57.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.690 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.699706026999998, 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:09:57.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:09:57.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.695 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.699686456000002, 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:09:57.696 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:09:57.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:09:57.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:09:57.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:09:57.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:09:57.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:09:57.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:09:57.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:09:57.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:09:57.701 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:09:57.703 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:57.701+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:57.704 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:57.704 INFO:teuthology.orchestra.run.smithi092.stdout:3559 punch from 0x8c69c00 to 0x8c78800, (0xec00 bytes) 2024-09-15T00:09:57.704 INFO:teuthology.orchestra.run.smithi092.stdout:3560 read 0x6beba00 thru 0x6bed098 (0x1699 bytes) 2024-09-15T00:09:57.704 INFO:teuthology.orchestra.run.smithi092.stdout:3561 write 0x8aa1c00 thru 0x8aa8345 (0x6746 bytes) 2024-09-15T00:09:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:3563 read 0x4048a00 thru 0x40552cc (0xc8cd bytes) 2024-09-15T00:09:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:3566 punch from 0x6434e00 to 0x643fc00, (0xae00 bytes) 2024-09-15T00:09:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:3567 write 0x31cd000 thru 0x31cff62 (0x2f63 bytes) 2024-09-15T00:09:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:3569 trunc from 0xcaf9800 to 0xe886200 2024-09-15T00:09:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:3570 write 0x98fb200 thru 0x98fb7ec (0x5ed bytes) 2024-09-15T00:09:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:3571 punch from 0x27ec400 to 0x27ec800, (0x400 bytes) 2024-09-15T00:09:57.706 INFO:teuthology.orchestra.run.smithi092.stdout:3572 read 0x166fc00 thru 0x1679caf (0xa0b0 bytes) 2024-09-15T00:09:57.717 INFO:teuthology.orchestra.run.smithi092.stdout:3573 read 0xb586a00 thru 0xb58c993 (0x5f94 bytes) 2024-09-15T00:09:57.718 INFO:teuthology.orchestra.run.smithi092.stdout:3574 write 0xcf48c00 thru 0xcf4fc64 (0x7065 bytes) 2024-09-15T00:09:57.718 INFO:teuthology.orchestra.run.smithi092.stdout:3575 read 0x5ad9800 thru 0x5ada0f2 (0x8f3 bytes) 2024-09-15T00:09:57.733 INFO:teuthology.orchestra.run.smithi092.stdout:3576 trunc from 0xe886200 to 0x1c37c00 2024-09-15T00:09:57.786 INFO:teuthology.orchestra.run.smithi092.stdout:3581 write 0x5d12800 thru 0x5d1a98f (0x8190 bytes) 2024-09-15T00:09:57.791 INFO:teuthology.orchestra.run.smithi092.stdout:3584 write 0x678a00 thru 0x680f11 (0x8512 bytes) 2024-09-15T00:09:57.793 INFO:teuthology.orchestra.run.smithi092.stdout:3585 write 0x3becc00 thru 0x3bf9c81 (0xd082 bytes) 2024-09-15T00:09:57.793 INFO:teuthology.orchestra.run.smithi092.stdout:3586 read 0x2d55000 thru 0x2d5e17e (0x917f bytes) 2024-09-15T00:09:57.793 INFO:teuthology.orchestra.run.smithi092.stdout:3588 clone 27 order 18 su 4096 sc 7 2024-09-15T00:09:57.803 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:57.801+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:57.820 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:57.820 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:57.820 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:09:57.821 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:57.829 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:57.841 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:09:57.904 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:57.902+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:57.936 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:09:57.936 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:09:57.936 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:09:57.936 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:09:57.937 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:09:57.945 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:09:57.979 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:09:57.997 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:09:57.997 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:09:58.005 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:58.004+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:58.016 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:09:58.016 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:09:58.036 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:09:58.036 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:09:58.054 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:09:58.054 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:09:58.064 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:58.064 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:58.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:09:58.074 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:09:58.074 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:09:58.076 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:09:58.076 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:09:58.092 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone26 from 0x5d1aa00 (overlap 0x1c37c00) to 0x2d2600 2024-09-15T00:09:58.107 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:58.105+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:58.207 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:09:58.206+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:09:58.230 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:58.230 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:58.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:09:58.291 INFO:teuthology.orchestra.run.smithi027.stdout:get_pg_num_history pg_num_history pg_num_history(e55 pg_nums {1={12=1},2={15=8},3={20=32}} deleted_pools ) 2024-09-15T00:09:58.291 INFO:teuthology.orchestra.run.smithi027.stdout:Importing pgid 3.c 2024-09-15T00:09:58.291 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:3005b819:::rbd_object_map.11291357fc5b:head# 2024-09-15T00:09:58.292 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:09:58.292 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:302ce972:::rbd_data.1129ebdcc4e7.000000000000000f:head# 2024-09-15T00:09:58.292 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:09:58.293 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:307e958b:::rbd_data.11296e862a2c.000000000000005f:c# 2024-09-15T00:09:58.295 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:307e958b:::rbd_data.11296e862a2c.000000000000005f:head# 2024-09-15T00:09:58.295 INFO:teuthology.orchestra.run.smithi027.stdout:snapset c=[]:{c=[c]} 2024-09-15T00:09:58.296 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:30d5d670:::rbd_data.11293dff0670.0000000000000007:e# 2024-09-15T00:09:58.298 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:30d5d670:::rbd_data.11293dff0670.0000000000000007:head# 2024-09-15T00:09:58.298 INFO:teuthology.orchestra.run.smithi027.stdout:snapset e=[]:{e=[e]} 2024-09-15T00:09:58.298 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:3121db20:::rbd_data.1129aa143e51.000000000000005d:5# 2024-09-15T00:09:58.299 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:3121db20:::rbd_data.1129aa143e51.000000000000005d:head# 2024-09-15T00:09:58.299 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 5=[]:{5=[5]} 2024-09-15T00:09:58.299 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:32606b9a:::rbd_object_map.1129abeafde8:head# 2024-09-15T00:09:58.299 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:09:58.299 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:33bd27dc:::rbd_data.11293dff0670.0000000000000024:e# 2024-09-15T00:09:58.300 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:33bd27dc:::rbd_data.11293dff0670.0000000000000024:head# 2024-09-15T00:09:58.300 INFO:teuthology.orchestra.run.smithi027.stdout:snapset e=[]:{e=[e]} 2024-09-15T00:09:58.301 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:340739f1:::rbd_data.112927a1b33d.0000000000000008:head# 2024-09-15T00:09:58.301 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:09:58.301 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:34ae4a59:::rbd_data.112910301a74.000000000000000f:head# 2024-09-15T00:09:58.302 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:09:58.304 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:34b47bce:::rbd_data.1129aa143e51.000000000000005c:5# 2024-09-15T00:09:58.304 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:34b47bce:::rbd_data.1129aa143e51.000000000000005c:head# 2024-09-15T00:09:58.304 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 5=[]:{5=[5]} 2024-09-15T00:09:58.305 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:34b73b99:::rbd_data.11298f711a6e.000000000000000b:head# 2024-09-15T00:09:58.305 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:09:58.305 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:35432b3e:::rbd_data.11291ce3fc54.0000000000000017:6# 2024-09-15T00:09:58.306 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:35432b3e:::rbd_data.11291ce3fc54.0000000000000017:head# 2024-09-15T00:09:58.306 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 6=[]:{6=[6]} 2024-09-15T00:09:58.307 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:355b8a9d:::rbd_data.1129aa143e51.0000000000000067:head# 2024-09-15T00:09:58.307 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 5=[]:{} 2024-09-15T00:09:58.307 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:3560610b:::rbd_data.112981949474.0000000000000043:head# 2024-09-15T00:09:58.307 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:09:58.307 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:37551d32:::rbd_data.11296e862a2c.000000000000005b:c# 2024-09-15T00:09:58.308 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:58.306+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:58.309 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:37551d32:::rbd_data.11296e862a2c.000000000000005b:head# 2024-09-15T00:09:58.309 INFO:teuthology.orchestra.run.smithi027.stdout:snapset c=[]:{c=[c]} 2024-09-15T00:09:58.310 INFO:teuthology.orchestra.run.smithi027.stdout:write_pg epoch 55 info 3.c( v 55'276 (0'0,55'276] local-lis/les=20/21 n=22 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-15T00:09:58.311 INFO:teuthology.orchestra.run.smithi027.stdout:Import successful 2024-09-15T00:09:58.352 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:58.352 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:58.361 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:09:58.409 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:58.408+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:58.471 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:58.471 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:58.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:09:58.522 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:58.521+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:58.593 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:58.593 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:58.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:09:58.603 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-15T00:09:58.645 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:58.644+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:58.711 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:58.710+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:58.720 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:58.720 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:58.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:09:58.814 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:58.813+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:58.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /var/log/ceph/exp.3.c.0 2024-09-15T00:09:58.901 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:58.901 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:58.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:09:58.953 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:58.951+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:59.015 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:59.014+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:59.077 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:59.077 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:59.086 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:09:59.116 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:59.115+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:59.121 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:09:59.188 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:59.189 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:59.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:09:59.240 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:59.239+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:59.265 INFO:teuthology.orchestra.run.smithi092.stdout:3589 read 0x5612a00 thru 0x561802d (0x562e bytes) 2024-09-15T00:09:59.271 INFO:teuthology.orchestra.run.smithi092.stdout:3590 trunc from 0x5d1aa00 to 0xa9faa00 2024-09-15T00:09:59.284 INFO:teuthology.orchestra.run.smithi092.stdout:3592 read 0x8dfb400 thru 0x8e0a9e0 (0xf5e1 bytes) 2024-09-15T00:09:59.286 INFO:teuthology.orchestra.run.smithi092.stdout:3593 read 0x6396c00 thru 0x639b6a7 (0x4aa8 bytes) 2024-09-15T00:09:59.286 INFO:teuthology.orchestra.run.smithi092.stdout:3594 punch from 0x6bc1200 to 0x6bcf200, (0xe000 bytes) 2024-09-15T00:09:59.298 INFO:teuthology.orchestra.run.smithi092.stdout:3595 trunc from 0xa9faa00 to 0x102b200 2024-09-15T00:09:59.306 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:59.307 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:59.317 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:09:59.315+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:09:59.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:09:59.367 INFO:teuthology.orchestra.run.smithi092.stdout:3597 punch from 0x1a5200 to 0x1a7600, (0x2400 bytes) 2024-09-15T00:09:59.403 INFO:teuthology.orchestra.run.smithi092.stdout:3598 write 0x52be000 thru 0x52c7098 (0x9099 bytes) 2024-09-15T00:09:59.403 INFO:teuthology.orchestra.run.smithi092.stdout:3599 punch from 0x13ba400 to 0x13c2200, (0x7e00 bytes) 2024-09-15T00:09:59.418 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:59.417+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:59.435 INFO:teuthology.orchestra.run.smithi092.stdout:3600 write 0xb641400 thru 0xb64ad95 (0x9996 bytes) 2024-09-15T00:09:59.437 INFO:teuthology.orchestra.run.smithi092.stdout:3601 read 0x45da600 thru 0x45e38a4 (0x92a5 bytes) 2024-09-15T00:09:59.437 INFO:teuthology.orchestra.run.smithi092.stdout:3602 punch from 0x1f9ec00 to 0x1fa2e00, (0x4200 bytes) 2024-09-15T00:09:59.439 INFO:teuthology.orchestra.run.smithi092.stdout:3603 read 0x935a000 thru 0x9366f63 (0xcf64 bytes) 2024-09-15T00:09:59.439 INFO:teuthology.orchestra.run.smithi092.stdout:3604 read 0x46e3800 thru 0x46ed895 (0xa096 bytes) 2024-09-15T00:09:59.441 INFO:teuthology.orchestra.run.smithi092.stdout:3605 trunc from 0xb64ae00 to 0xd155200 2024-09-15T00:09:59.470 INFO:teuthology.orchestra.run.smithi092.stdout:3606 punch from 0x9424400 to 0x9428400, (0x4000 bytes) 2024-09-15T00:09:59.490 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:59.490 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:59.496 INFO:teuthology.orchestra.run.smithi092.stdout:3607 write 0xec84c00 thru 0xec8c741 (0x7b42 bytes) 2024-09-15T00:09:59.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:09:59.513 INFO:teuthology.orchestra.run.smithi092.stdout:3608 trunc from 0xec8c800 to 0xdd7e00 2024-09-15T00:09:59.549 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:59.547+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:59.597 INFO:teuthology.orchestra.run.smithi092.stdout:3610 read 0xac2600 thru 0xac8009 (0x5a0a bytes) 2024-09-15T00:09:59.598 INFO:teuthology.orchestra.run.smithi092.stdout:3612 read 0x9a9400 thru 0x9b120e (0x7e0f bytes) 2024-09-15T00:09:59.600 INFO:teuthology.orchestra.run.smithi092.stdout:3614 punch from 0x2aa00 to 0x37200, (0xc800 bytes) 2024-09-15T00:09:59.620 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:59.619+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:59.636 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:59.636 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:59.642 INFO:teuthology.orchestra.run.smithi092.stdout:3619 write 0xafcc000 thru 0xafd6e65 (0xae66 bytes) 2024-09-15T00:09:59.642 INFO:teuthology.orchestra.run.smithi092.stdout:3620 read 0x2a3e00 thru 0x2a710a (0x330b bytes) 2024-09-15T00:09:59.643 INFO:teuthology.orchestra.run.smithi092.stdout:3622 write 0xcfe600 thru 0xd0a0fc (0xbafd bytes) 2024-09-15T00:09:59.644 INFO:teuthology.orchestra.run.smithi092.stdout:3623 write 0x1fac400 thru 0x1fb53f0 (0x8ff1 bytes) 2024-09-15T00:09:59.644 INFO:teuthology.orchestra.run.smithi092.stdout:3625 read 0x5db6200 thru 0x5db6a18 (0x819 bytes) 2024-09-15T00:09:59.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:09:59.650 INFO:teuthology.orchestra.run.smithi092.stdout:3627 trunc from 0xafd7000 to 0x5897800 2024-09-15T00:09:59.685 INFO:teuthology.orchestra.run.smithi092.stdout:3630 punch from 0x102cc00 to 0x1039400, (0xc800 bytes) 2024-09-15T00:09:59.685 INFO:teuthology.orchestra.run.smithi092.stdout:3632 trunc from 0x5897800 to 0x4dc0800 2024-09-15T00:09:59.721 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:09:59.720+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:09:59.729 INFO:teuthology.orchestra.run.smithi092.stdout:3633 write 0xbd64600 thru 0xbd717f9 (0xd1fa bytes) 2024-09-15T00:09:59.740 INFO:teuthology.orchestra.run.smithi092.stdout:3634 trunc from 0xbd71800 to 0x1eeb400 2024-09-15T00:09:59.801 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:59.801 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:59.809 INFO:teuthology.orchestra.run.smithi092.stdout:3635 write 0xd0a5a00 thru 0xd0aa30f (0x4910 bytes) 2024-09-15T00:09:59.809 INFO:teuthology.orchestra.run.smithi092.stdout:3636 read 0x51c9200 thru 0x51d0b5b (0x795c bytes) 2024-09-15T00:09:59.811 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:09:59.812 INFO:teuthology.orchestra.run.smithi092.stdout:3637 trunc from 0xd0aa400 to 0xe1f2600 2024-09-15T00:09:59.833 INFO:teuthology.orchestra.run.smithi092.stdout:3638 punch from 0xaeae200 to 0xaeba000, (0xbe00 bytes) 2024-09-15T00:09:59.833 INFO:teuthology.orchestra.run.smithi092.stdout:3639 write 0x8a89800 thru 0x8a93806 (0xa007 bytes) 2024-09-15T00:09:59.833 INFO:teuthology.orchestra.run.smithi092.stdout:3640 punch from 0x2220600 to 0x2227000, (0x6a00 bytes) 2024-09-15T00:09:59.840 INFO:teuthology.orchestra.run.smithi092.stdout:3641 trunc from 0xe1f2600 to 0x8ed7200 2024-09-15T00:09:59.856 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:09:59.855+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:09:59.878 INFO:teuthology.orchestra.run.smithi092.stdout:3642 write 0x1176400 thru 0x117bc75 (0x5876 bytes) 2024-09-15T00:09:59.879 INFO:teuthology.orchestra.run.smithi092.stdout:3644 punch from 0x20da00 to 0x20e800, (0xe00 bytes) 2024-09-15T00:09:59.909 INFO:teuthology.orchestra.run.smithi092.stdout:3647 write 0x9936c00 thru 0x994634c (0xf74d bytes) 2024-09-15T00:09:59.910 INFO:teuthology.orchestra.run.smithi092.stdout:3649 read 0x3ef6400 thru 0x3ef8cfa (0x28fb bytes) 2024-09-15T00:09:59.910 INFO:teuthology.orchestra.run.smithi092.stdout:3650 punch from 0x80d3e00 to 0x80da400, (0x6600 bytes) 2024-09-15T00:09:59.910 INFO:teuthology.orchestra.run.smithi092.stdout:3651 read 0x844a600 thru 0x844c9f3 (0x23f4 bytes) 2024-09-15T00:09:59.910 INFO:teuthology.orchestra.run.smithi092.stdout:3653 write 0x5aed800 thru 0x5afd369 (0xfb6a bytes) 2024-09-15T00:09:59.911 INFO:teuthology.orchestra.run.smithi092.stdout:3654 read 0x3e1a800 thru 0x3e28bfe (0xe3ff bytes) 2024-09-15T00:09:59.917 INFO:teuthology.orchestra.run.smithi092.stdout:3656 trunc from 0x9946400 to 0x4109800 2024-09-15T00:09:59.923 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:09:59.921+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:09:59.927 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:09:59.927 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:09:59.937 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:09:59.973 INFO:teuthology.orchestra.run.smithi092.stdout:3661 write 0xbb56400 thru 0xbb5d4e5 (0x70e6 bytes) 2024-09-15T00:09:59.978 INFO:teuthology.orchestra.run.smithi092.stdout:3662 punch from 0x9d73a00 to 0x9d7b800, (0x7e00 bytes) 2024-09-15T00:09:59.987 INFO:teuthology.orchestra.run.smithi092.stdout:3663 trunc from 0xbb5d600 to 0x49a2e00 2024-09-15T00:10:00.014 INFO:teuthology.orchestra.run.smithi092.stdout:3665 read 0x40ce400 thru 0x40d3ff8 (0x5bf9 bytes) 2024-09-15T00:10:00.015 INFO:teuthology.orchestra.run.smithi092.stdout:3669 write 0x1a5dc00 thru 0x1a69206 (0xb607 bytes) 2024-09-15T00:10:00.015 INFO:teuthology.orchestra.run.smithi092.stdout:3670 read 0x19dfc00 thru 0x19eca7f (0xce80 bytes) 2024-09-15T00:10:00.015 INFO:teuthology.orchestra.run.smithi092.stdout:3671 write 0x3463600 thru 0x3465629 (0x202a bytes) 2024-09-15T00:10:00.023 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:00.022+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:10:00.048 INFO:teuthology.orchestra.run.smithi092.stdout:3674 write 0xda2ca00 thru 0xda39c0f (0xd210 bytes) 2024-09-15T00:10:00.049 INFO:teuthology.orchestra.run.smithi092.stdout:3675 read 0xcc34800 thru 0xcc3e941 (0xa142 bytes) 2024-09-15T00:10:00.050 INFO:teuthology.orchestra.run.smithi092.stdout:3679 write 0x716ac00 thru 0x716ca16 (0x1e17 bytes) 2024-09-15T00:10:00.050 INFO:teuthology.orchestra.run.smithi092.stdout:3680 read 0xb4b2400 thru 0xb4c14ce (0xf0cf bytes) 2024-09-15T00:10:00.050 INFO:teuthology.orchestra.run.smithi092.stdout:3683 write 0xd264e00 thru 0xd26b85c (0x6a5d bytes) 2024-09-15T00:10:00.050 INFO:teuthology.orchestra.run.smithi092.stdout:3685 write 0xb270000 thru 0xb274a5c (0x4a5d bytes) 2024-09-15T00:10:00.050 INFO:teuthology.orchestra.run.smithi092.stdout:3687 read 0x78edc00 thru 0x78f5548 (0x7949 bytes) 2024-09-15T00:10:00.050 INFO:teuthology.orchestra.run.smithi092.stdout:3688 read 0x3705c00 thru 0x3707bce (0x1fcf bytes) 2024-09-15T00:10:00.050 INFO:teuthology.orchestra.run.smithi092.stdout:3689 write 0x3d96000 thru 0x3da1221 (0xb222 bytes) 2024-09-15T00:10:00.050 INFO:teuthology.orchestra.run.smithi092.stdout:3691 punch from 0x8745c00 to 0x874fa00, (0x9e00 bytes) 2024-09-15T00:10:00.051 INFO:teuthology.orchestra.run.smithi092.stdout:3695 punch from 0x2d9f200 to 0x2dada00, (0xe800 bytes) 2024-09-15T00:10:00.051 INFO:teuthology.orchestra.run.smithi092.stdout:3697 write 0x9d63400 thru 0x9d6772a (0x432b bytes) 2024-09-15T00:10:00.051 INFO:teuthology.orchestra.run.smithi092.stdout:3698 punch from 0x8747600 to 0x874f800, (0x8200 bytes) 2024-09-15T00:10:00.077 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:00.088 INFO:teuthology.orchestra.run.smithi092.stdout:3699 read 0x1fbf000 thru 0x1fc7554 (0x8555 bytes) 2024-09-15T00:10:00.088 INFO:teuthology.orchestra.run.smithi092.stdout:3701 punch from 0x94d1200 to 0x94dec00, (0xda00 bytes) 2024-09-15T00:10:00.088 INFO:teuthology.orchestra.run.smithi092.stdout:3702 write 0xbb29200 thru 0xbb3689f (0xd6a0 bytes) 2024-09-15T00:10:00.102 INFO:teuthology.orchestra.run.smithi092.stdout:3704 trunc from 0xda39e00 to 0x1e9ca00 2024-09-15T00:10:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:00.109 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:00.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:00.124 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:00.123+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:00.176 INFO:teuthology.orchestra.run.smithi092.stdout:3707 read 0xd45600 thru 0xd493bb (0x3dbc bytes) 2024-09-15T00:10:00.178 INFO:teuthology.orchestra.run.smithi092.stdout:3708 read 0x87e200 thru 0x87ee96 (0xc97 bytes) 2024-09-15T00:10:00.202 INFO:teuthology.orchestra.run.smithi092.stdout:3709 write 0xdae1600 thru 0xdae9145 (0x7b46 bytes) 2024-09-15T00:10:00.203 INFO:teuthology.orchestra.run.smithi092.stdout:3712 write 0x7414600 thru 0x7418889 (0x428a bytes) 2024-09-15T00:10:00.203 INFO:teuthology.orchestra.run.smithi092.stdout:3713 read 0x2c02600 thru 0x2c063ec (0x3ded bytes) 2024-09-15T00:10:00.203 INFO:teuthology.orchestra.run.smithi092.stdout:3717 read 0x66efa00 thru 0x66fc449 (0xca4a bytes) 2024-09-15T00:10:00.203 INFO:teuthology.orchestra.run.smithi092.stdout:3722 punch from 0x3811800 to 0x3812600, (0xe00 bytes) 2024-09-15T00:10:00.204 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:00.204 INFO:teuthology.orchestra.run.smithi092.stdout:3723 write 0xfe6c00 thru 0xfe89c4 (0x1dc5 bytes) 2024-09-15T00:10:00.205 INFO:teuthology.orchestra.run.smithi092.stdout:3724 write 0x1d45600 thru 0x1d48ab6 (0x34b7 bytes) 2024-09-15T00:10:00.205 INFO:teuthology.orchestra.run.smithi092.stdout:3725 punch from 0x7d5a000 to 0x7d5e600, (0x4600 bytes) 2024-09-15T00:10:00.205 INFO:teuthology.orchestra.run.smithi092.stdout:3729 write 0x456a000 thru 0x457814e (0xe14f bytes) 2024-09-15T00:10:00.206 INFO:teuthology.orchestra.run.smithi092.stdout:3730 punch from 0xa4eba00 to 0xa4ee800, (0x2e00 bytes) 2024-09-15T00:10:00.219 INFO:teuthology.orchestra.run.smithi092.stdout:3733 trunc from 0xdae9200 to 0x26a9e00 2024-09-15T00:10:00.226 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:00.224+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:00.238 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:00.239 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:00.273 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:00.290 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:00.291 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:00.300 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:00.300 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:00.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:00.300 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.270115474000001, 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:00.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:00.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:10:00.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:10:00.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.305 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.270095219000002, 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.306 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:00.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:00.310 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.270075472999999, 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.311 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:00.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.312 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:00.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:00.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:00.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:00.316 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.316 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.316 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.270055523, 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.317 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:00.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.270028022999998, 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:00.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:00.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:00.326 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.270011182000001, 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:00.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:00.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.331 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.269995183999999, 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.332 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:00.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:00.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:00.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.337 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.337 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.337 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:00.337 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.269972196000001, 2024-09-15T00:10:00.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:00.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:00.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.269946311999998, 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:00.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.269929885, 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:00.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:00.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:00.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:00.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:00.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:00.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:00.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:00.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:00.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:00.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.352 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.269915613999999, 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:00.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.269906296999999, 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.357 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:00.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.268237285000001, 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:00.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:00.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.268202221999999, 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:00.366 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.268177195, 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:00.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.268152886999999, 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.376 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:00.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2674 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3768320~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.266006540999999, 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059850591000000002, 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2674 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031671+0000", 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031682+0000", 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1248e-05 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031693+0000", 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1395e-05 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031705+0000", 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2106000000000001e-05 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091352+0000", 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059646421999999998 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091441+0000", 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9174999999999999e-05 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091521+0000", 2024-09-15T00:10:00.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9636999999999997e-05 2024-09-15T00:10:00.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.385 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.385 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.265960635999999, 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:10:00.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:10:00.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:10:00.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.390 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.265936503999999, 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.394 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.265907131999999, 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:00.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:00.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.399 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:00.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:00.402 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:00.402 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:00.402 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:00.403 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:00.404 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:00.404 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:00.399+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:00.481 INFO:teuthology.orchestra.run.smithi092.stdout:3734 trunc from 0x26a9e00 to 0x3758600 2024-09-15T00:10:00.496 INFO:teuthology.orchestra.run.smithi092.stdout:3739 read 0xf2a400 thru 0xf37041 (0xcc42 bytes) 2024-09-15T00:10:00.496 INFO:teuthology.orchestra.run.smithi092.stdout:3740 read 0x13e9000 thru 0x13ebf2a (0x2f2b bytes) 2024-09-15T00:10:00.496 INFO:teuthology.orchestra.run.smithi092.stdout:3741 punch from 0x2259200 to 0x2266c00, (0xda00 bytes) 2024-09-15T00:10:00.501 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:00.500+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:00.522 INFO:teuthology.orchestra.run.smithi092.stdout:3745 write 0xd69d200 thru 0xd69ea4c (0x184d bytes) 2024-09-15T00:10:00.522 INFO:teuthology.orchestra.run.smithi092.stdout:3746 read 0x2db2e00 thru 0x2dbc31e (0x951f bytes) 2024-09-15T00:10:00.523 INFO:teuthology.orchestra.run.smithi092.stdout:3750 write 0x2c22a00 thru 0x2c2e19e (0xb79f bytes) 2024-09-15T00:10:00.523 INFO:teuthology.orchestra.run.smithi092.stdout:3751 punch from 0x646b000 to 0x6473c00, (0x8c00 bytes) 2024-09-15T00:10:00.523 INFO:teuthology.orchestra.run.smithi092.stdout:3759 punch from 0xa257c00 to 0xa259800, (0x1c00 bytes) 2024-09-15T00:10:00.555 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:00.555 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:00.556 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:00.556 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:00.557 INFO:teuthology.orchestra.run.smithi092.stdout:3761 write 0xe914400 thru 0xe914eb6 (0xab7 bytes) 2024-09-15T00:10:00.557 INFO:teuthology.orchestra.run.smithi092.stdout:3762 write 0x263ac00 thru 0x263d585 (0x2986 bytes) 2024-09-15T00:10:00.557 INFO:teuthology.orchestra.run.smithi092.stdout:3764 write 0x1b85c00 thru 0x1b94582 (0xe983 bytes) 2024-09-15T00:10:00.558 INFO:teuthology.orchestra.run.smithi092.stdout:3765 punch from 0x6333400 to 0x6337000, (0x3c00 bytes) 2024-09-15T00:10:00.558 INFO:teuthology.orchestra.run.smithi092.stdout:3769 write 0x41fce00 thru 0x4200e6a (0x406b bytes) 2024-09-15T00:10:00.558 INFO:teuthology.orchestra.run.smithi092.stdout:3770 punch from 0x1c78c00 to 0x1c7ca00, (0x3e00 bytes) 2024-09-15T00:10:00.559 INFO:teuthology.orchestra.run.smithi092.stdout:3771 read 0xce0c200 thru 0xce0ecb8 (0x2ab9 bytes) 2024-09-15T00:10:00.559 INFO:teuthology.orchestra.run.smithi092.stdout:3772 write 0x7d14e00 thru 0x7d1622b (0x142c bytes) 2024-09-15T00:10:00.565 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:00.566 INFO:teuthology.orchestra.run.smithi092.stdout:3775 trunc from 0xe915000 to 0x81faa00 2024-09-15T00:10:00.600 INFO:teuthology.orchestra.run.smithi092.stdout:3777 write 0x39f6e00 thru 0x3a03a36 (0xcc37 bytes) 2024-09-15T00:10:00.601 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:00.600+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:00.602 INFO:teuthology.orchestra.run.smithi092.stdout:3778 write 0x2f2da00 thru 0x2f3c85c (0xee5d bytes) 2024-09-15T00:10:00.602 INFO:teuthology.orchestra.run.smithi092.stdout:3780 punch from 0x7fd600 to 0x80a200, (0xcc00 bytes) 2024-09-15T00:10:00.605 INFO:teuthology.orchestra.run.smithi092.stdout:3781 punch from 0x6088c00 to 0x6092200, (0x9600 bytes) 2024-09-15T00:10:00.671 INFO:teuthology.orchestra.run.smithi092.stdout:3782 write 0xbb57600 thru 0xbb5a319 (0x2d1a bytes) 2024-09-15T00:10:00.684 INFO:teuthology.orchestra.run.smithi092.stdout:3783 trunc from 0xbb5a400 to 0x55a7400 2024-09-15T00:10:00.703 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:00.702+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:00.716 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:00.716 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:00.716 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:00.716 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:00.717 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:00.722 INFO:teuthology.orchestra.run.smithi092.stdout:3785 write 0x5443c00 thru 0x5452fb3 (0xf3b4 bytes) 2024-09-15T00:10:00.725 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:00.733 INFO:teuthology.orchestra.run.smithi092.stdout:3786 trunc from 0x55a7400 to 0xe742e00 2024-09-15T00:10:00.756 INFO:teuthology.orchestra.run.smithi092.stdout:3789 read 0x157dc00 thru 0x158a44b (0xc84c bytes) 2024-09-15T00:10:00.757 INFO:teuthology.orchestra.run.smithi092.stdout:3790 write 0xabc7c00 thru 0xabd67a7 (0xeba8 bytes) 2024-09-15T00:10:00.758 INFO:teuthology.orchestra.run.smithi092.stdout:3791 punch from 0x557f000 to 0x558b200, (0xc200 bytes) 2024-09-15T00:10:00.767 INFO:teuthology.orchestra.run.smithi092.stdout:3793 trunc from 0xe742e00 to 0x6e8a000 2024-09-15T00:10:00.798 INFO:teuthology.orchestra.run.smithi092.stdout:3794 write 0x8fe2000 thru 0x8fecbf6 (0xabf7 bytes) 2024-09-15T00:10:00.799 INFO:teuthology.orchestra.run.smithi092.stdout:3796 write 0x128f600 thru 0x12949c0 (0x53c1 bytes) 2024-09-15T00:10:00.801 INFO:teuthology.orchestra.run.smithi092.stdout:3797 trunc from 0x8fecc00 to 0x7287e00 2024-09-15T00:10:00.804 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:00.802+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:00.829 INFO:teuthology.orchestra.run.smithi092.stdout:3798 read 0x70fb200 thru 0x710a7a0 (0xf5a1 bytes) 2024-09-15T00:10:00.843 INFO:teuthology.orchestra.run.smithi092.stdout:3801 write 0xa5b4000 thru 0xa5b8ee4 (0x4ee5 bytes) 2024-09-15T00:10:00.847 INFO:teuthology.orchestra.run.smithi092.stdout:3803 trunc from 0xa5b9000 to 0xd8b5c00 2024-09-15T00:10:00.863 INFO:teuthology.orchestra.run.smithi092.stdout:3804 punch from 0x4203e00 to 0x4211800, (0xda00 bytes) 2024-09-15T00:10:00.877 INFO:teuthology.orchestra.run.smithi092.stdout:3805 trunc from 0xd8b5c00 to 0x6062800 2024-09-15T00:10:00.880 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.852694984999999, 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.881 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:00.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.852673067000001, 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.886 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:00.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:00.890 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.852651795, 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:00.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:00.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.850910535000001, 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.895 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:00.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.850888629, 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:00.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.850866537000002, 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:00.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.850834798000001, 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.909 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:00.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:00.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.913 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.850536595000001, 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:00.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:00.918 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.850502890000001, 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:00.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:00.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.846200817, 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.923 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:00.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.845875947, 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.928 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:00.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.845465541999999, 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.932 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:00.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.845410639000001, 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:00.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.845378604, 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:00.941 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:00.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:00.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.843900774000002, 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.946 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:00.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.843879058999999, 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.950 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:00.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.843857580000002, 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:00.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:00.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:00.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:00.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.841844475999999, 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.960 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:00.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:00.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.841824835000001, 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.965 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:00.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.841778287, 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:00.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:00.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:00.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:00.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:00.973 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:00.975 INFO:teuthology.orchestra.run.smithi092.stdout:3808 write 0x40ca800 thru 0x40d1e6e (0x766f bytes) 2024-09-15T00:10:00.975 INFO:teuthology.orchestra.run.smithi092.stdout:3810 write 0xa3e8400 thru 0xa3f5264 (0xce65 bytes) 2024-09-15T00:10:00.975 INFO:teuthology.orchestra.run.smithi092.stdout:3813 trunc from 0xa3f5400 to 0x40d2600 2024-09-15T00:10:00.976 INFO:teuthology.orchestra.run.smithi092.stdout:3818 trunc from 0x40d2600 to 0x53b8400 2024-09-15T00:10:00.976 INFO:teuthology.orchestra.run.smithi092.stdout:3819 punch from 0x61f200 to 0x62a400, (0xb200 bytes) 2024-09-15T00:10:00.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:00.976 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:00.974+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:00.983 INFO:teuthology.orchestra.run.smithi092.stdout:3820 write 0xd24a00 thru 0xd3381c (0xee1d bytes) 2024-09-15T00:10:01.027 INFO:teuthology.orchestra.run.smithi092.stdout:3821 write 0x9cb9e00 thru 0x9cc4b8c (0xad8d bytes) 2024-09-15T00:10:01.034 INFO:teuthology.orchestra.run.smithi092.stdout:3823 punch from 0x7b89000 to 0x7b8c400, (0x3400 bytes) 2024-09-15T00:10:01.036 INFO:teuthology.orchestra.run.smithi092.stdout:3824 write 0x1179200 thru 0x11848e0 (0xb6e1 bytes) 2024-09-15T00:10:01.036 INFO:teuthology.orchestra.run.smithi092.stdout:3825 read 0xfd9800 thru 0xfda315 (0xb16 bytes) 2024-09-15T00:10:01.064 INFO:teuthology.orchestra.run.smithi092.stdout:3828 write 0xc6ad000 thru 0xc6ae188 (0x1189 bytes) 2024-09-15T00:10:01.064 INFO:teuthology.orchestra.run.smithi092.stdout:3831 read 0xa9af400 thru 0xa9bbd4a (0xc94b bytes) 2024-09-15T00:10:01.064 INFO:teuthology.orchestra.run.smithi092.stdout:3832 read 0x70a2200 thru 0x70a9832 (0x7633 bytes) 2024-09-15T00:10:01.064 INFO:teuthology.orchestra.run.smithi092.stdout:3833 write 0x183d400 thru 0x184c6d8 (0xf2d9 bytes) 2024-09-15T00:10:01.065 INFO:teuthology.orchestra.run.smithi092.stdout:3839 write 0xb29600 thru 0xb2a81c (0x121d bytes) 2024-09-15T00:10:01.076 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:01.074+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:01.095 INFO:teuthology.orchestra.run.smithi092.stdout:3841 write 0xdcada00 thru 0xdcb000f (0x2610 bytes) 2024-09-15T00:10:01.096 INFO:teuthology.orchestra.run.smithi092.stdout:3843 read 0xb8f4800 thru 0xb90384e (0xf04f bytes) 2024-09-15T00:10:01.096 INFO:teuthology.orchestra.run.smithi092.stdout:3847 read 0x4493800 thru 0x449700f (0x3810 bytes) 2024-09-15T00:10:01.096 INFO:teuthology.orchestra.run.smithi092.stdout:3848 write 0x8f7ba00 thru 0x8f7da1e (0x201f bytes) 2024-09-15T00:10:01.096 INFO:teuthology.orchestra.run.smithi092.stdout:3853 punch from 0x3b00a00 to 0x3b04400, (0x3a00 bytes) 2024-09-15T00:10:01.098 INFO:teuthology.orchestra.run.smithi092.stdout:3855 read 0x406e800 thru 0x40703d6 (0x1bd7 bytes) 2024-09-15T00:10:01.098 INFO:teuthology.orchestra.run.smithi092.stdout:3856 write 0x6362c00 thru 0x636e194 (0xb595 bytes) 2024-09-15T00:10:01.098 INFO:teuthology.orchestra.run.smithi092.stdout:3857 read 0x1fdd400 thru 0x1fdffc1 (0x2bc2 bytes) 2024-09-15T00:10:01.098 INFO:teuthology.orchestra.run.smithi092.stdout:3858 read 0x497c000 thru 0x4987bed (0xbbee bytes) 2024-09-15T00:10:01.098 INFO:teuthology.orchestra.run.smithi092.stdout:3863 read 0x2d3d400 thru 0x2d42821 (0x5422 bytes) 2024-09-15T00:10:01.099 INFO:teuthology.orchestra.run.smithi092.stdout:3864 punch from 0x8c0a600 to 0x8c0ac00, (0x600 bytes) 2024-09-15T00:10:01.099 INFO:teuthology.orchestra.run.smithi092.stdout:3865 read 0xbcab800 thru 0xbcacf84 (0x1785 bytes) 2024-09-15T00:10:01.099 INFO:teuthology.orchestra.run.smithi092.stdout:3867 punch from 0x1916e00 to 0x1919e00, (0x3000 bytes) 2024-09-15T00:10:01.113 INFO:teuthology.orchestra.run.smithi092.stdout:3868 trunc from 0xdcb0200 to 0x24df600 2024-09-15T00:10:01.120 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:8599 3.1f 3:fac72e80:::rbd_data.1129203d0835.0000000000000004:1c [read 262144~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:01.119384+0000", 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.000289938, 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031171600000000002, 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:01.121 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 8599 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119384+0000", 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119384+0000", 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119385+0000", 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2999999999999999e-07 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119390+0000", 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6890000000000003e-06 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119402+0000", 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2354999999999999e-05 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119409+0000", 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9659999999999999e-06 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119468+0000", 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9382999999999997e-05 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119555+0000", 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6335000000000006e-05 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119696+0000", 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141058 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:8587 3.11 3:8c0bfbfc:::rbd_data.112929d537d9.0000000000000233:head [set-alloc-hint object_size 262144 write_size 262144,write 5632~1536 in=1536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:10:01.125 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:01.115629+0000", 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0040445630000000002, 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041568459999999996, 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 8587 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115629+0000", 2024-09-15T00:10:01.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115629+0000", 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115631+0000", 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0449999999999999e-06 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115637+0000", 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4119999999999998e-06 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115650+0000", 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3043e-05 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115655+0000", 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3809999999999998e-06 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115852+0000", 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000196825 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115968+0000", 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115664 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:10:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.116103+0000", 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134945 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.119225+0000", 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031222759999999998 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:8588 3.a 3:53c6a300:::rbd_data.112929d537d9.0000000000000065:head [set-alloc-hint object_size 262144 write_size 262144,write 89600~512 in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:01.115683+0000", 2024-09-15T00:10:01.130 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0039909960000000001, 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041501990000000002, 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 8588 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115683+0000", 2024-09-15T00:10:01.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115683+0000", 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115684+0000", 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1999999999999999e-07 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115687+0000", 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.709e-06 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115695+0000", 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6369999999999999e-06 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115701+0000", 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4750000000000001e-06 2024-09-15T00:10:01.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115771+0000", 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0587999999999996e-05 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.115909+0000", 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013758499999999999 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-15T00:10:01.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.116079+0000", 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017051600000000001 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:01.118944+0000", 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028648290000000002 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 3 2024-09-15T00:10:01.135 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:01.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:01.166 INFO:teuthology.orchestra.run.smithi092.stdout:3870 read 0xb81e00 thru 0xb852a0 (0x34a1 bytes) 2024-09-15T00:10:01.173 INFO:teuthology.orchestra.run.smithi092.stdout:3871 read 0xd6f000 thru 0xd7d51f (0xe520 bytes) 2024-09-15T00:10:01.178 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:01.176+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:01.192 INFO:teuthology.orchestra.run.smithi092.stdout:3872 write 0x8f6a200 thru 0x8f706e3 (0x64e4 bytes) 2024-09-15T00:10:01.197 INFO:teuthology.orchestra.run.smithi092.stdout:3873 read 0x5a42800 thru 0x5a4c443 (0x9c44 bytes) 2024-09-15T00:10:01.197 INFO:teuthology.orchestra.run.smithi092.stdout:3875 trunc from 0x8f70800 to 0x59f5200 2024-09-15T00:10:01.229 INFO:teuthology.orchestra.run.smithi092.stdout:3877 read 0x52d7a00 thru 0x52da96f (0x2f70 bytes) 2024-09-15T00:10:01.230 INFO:teuthology.orchestra.run.smithi092.stdout:3878 trunc from 0x59f5200 to 0x6a22800 2024-09-15T00:10:01.239 INFO:teuthology.orchestra.run.smithi092.stdout:3879 write 0x3d0e400 thru 0x3d13b05 (0x5706 bytes) 2024-09-15T00:10:01.245 INFO:teuthology.orchestra.run.smithi092.stdout:3881 punch from 0x671b600 to 0x6729a00, (0xe400 bytes) 2024-09-15T00:10:01.246 INFO:teuthology.orchestra.run.smithi092.stdout:3882 read 0x2053000 thru 0x205cd69 (0x9d6a bytes) 2024-09-15T00:10:01.246 INFO:teuthology.orchestra.run.smithi092.stdout:3883 punch from 0x36cb800 to 0x36cca00, (0x1200 bytes) 2024-09-15T00:10:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:3884 read 0x6906200 thru 0x690af5f (0x4d60 bytes) 2024-09-15T00:10:01.271 INFO:teuthology.orchestra.run.smithi092.stdout:3886 write 0xd44f800 thru 0xd457ffc (0x87fd bytes) 2024-09-15T00:10:01.277 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:01.276+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:01.282 INFO:teuthology.orchestra.run.smithi092.stdout:3888 trunc from 0xd458000 to 0x8546800 2024-09-15T00:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:01.310 INFO:teuthology.orchestra.run.smithi092.stdout:3889 punch from 0x731a800 to 0x7322000, (0x7800 bytes) 2024-09-15T00:10:01.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:01.318 INFO:teuthology.orchestra.run.smithi092.stdout:3890 read 0x928e00 thru 0x936b18 (0xdd19 bytes) 2024-09-15T00:10:01.322 INFO:teuthology.orchestra.run.smithi092.stdout:3891 write 0x2f800 thru 0x3a231 (0xaa32 bytes) 2024-09-15T00:10:01.327 INFO:teuthology.orchestra.run.smithi092.stdout:3892 trunc from 0x8546800 to 0x4bfac00 2024-09-15T00:10:01.352 INFO:teuthology.orchestra.run.smithi092.stdout:3895 punch from 0x83b600 to 0x83c000, (0xa00 bytes) 2024-09-15T00:10:01.360 INFO:teuthology.orchestra.run.smithi092.stdout:3897 trunc from 0x4bfac00 to 0xc93e00 2024-09-15T00:10:01.378 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:01.377+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:01.400 INFO:teuthology.orchestra.run.smithi092.stdout:3898 read 0x177e00 thru 0x186f99 (0xf19a bytes) 2024-09-15T00:10:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:3899 read 0x947600 thru 0x947965 (0x366 bytes) 2024-09-15T00:10:01.407 INFO:teuthology.orchestra.run.smithi092.stdout:3901 punch from 0x26b000 to 0x275c00, (0xac00 bytes) 2024-09-15T00:10:01.413 INFO:teuthology.orchestra.run.smithi092.stdout:3902 trunc from 0xc93e00 to 0x3c32e00 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4070 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086374294000002, 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02702953, 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4070 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350999+0000", 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351003+0000", 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351007+0000", 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6820000000000001e-06 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351010+0000", 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7269999999999999e-06 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377929+0000", 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026919887999999999 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377969+0000", 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9909e-05 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378028+0000", 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8578000000000001e-05 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.444 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4071 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086359414, 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027136008, 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4071 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351014+0000", 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351015+0000", 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7039999999999999e-06 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351020+0000", 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351023+0000", 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378053+0000", 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027029610999999999 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378094+0000", 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1826999999999998e-05 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378149+0000", 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4815000000000003e-05 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4072 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086334735000001, 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027223404999999999, 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4072 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-15T00:10:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351040+0000", 2024-09-15T00:10:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:10:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351046+0000", 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351051+0000", 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378171+0000", 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027120590999999999 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378209+0000", 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7308999999999998e-05 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378261+0000", 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2753999999999997e-05 2024-09-15T00:10:01.454 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4074 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086315839999997, 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027334509999999999, 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4074 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9200000000000002e-07 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351059+0000", 2024-09-15T00:10:01.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.012e-06 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351065+0000", 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2129999999999996e-06 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351070+0000", 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8470000000000003e-06 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378285+0000", 2024-09-15T00:10:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027214449000000002 2024-09-15T00:10:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378334+0000", 2024-09-15T00:10:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:10:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378391+0000", 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7046000000000002e-05 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4075 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086296969000003, 2024-09-15T00:10:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027546923000000001, 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4075 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351078+0000", 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.905e-06 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351085+0000", 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351090+0000", 2024-09-15T00:10:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378423+0000", 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027333051000000001 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378523+0000", 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9959999999999998e-05 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378623+0000", 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9962000000000003e-05 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4077 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086274285000002, 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027660619000000001, 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4077 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351099+0000", 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351101+0000", 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:10:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351107+0000", 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7980000000000002e-06 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351111+0000", 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:10:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378657+0000", 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027545463999999999 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378702+0000", 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5809999999999997e-05 2024-09-15T00:10:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378759+0000", 2024-09-15T00:10:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-15T00:10:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4078 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:10:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086246285000001, 2024-09-15T00:10:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027808405000000001, 2024-09-15T00:10:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4078 2024-09-15T00:10:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:10:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:10:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351127+0000", 2024-09-15T00:10:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351129+0000", 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351135+0000", 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3820000000000001e-06 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351141+0000", 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2259999999999996e-06 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378785+0000", 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027644592999999999 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378855+0000", 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0276000000000003e-05 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378935+0000", 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9361000000000001e-05 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4079 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086224647999998, 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027913778, 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4079 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351150+0000", 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9010000000000001e-06 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351157+0000", 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351162+0000", 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378958+0000", 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027795786999999999 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379004+0000", 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6124000000000001e-05 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379062+0000", 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7630999999999998e-05 2024-09-15T00:10:01.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4081 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086200967000003, 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016658, 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4081 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351174+0000", 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351181+0000", 2024-09-15T00:10:01.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9070000000000001e-06 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351186+0000", 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.651e-06 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379085+0000", 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027898757999999999 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379130+0000", 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5126000000000001e-05 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379188+0000", 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8548999999999998e-05 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4082 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086180913, 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028118457999999999, 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4082 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351194+0000", 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7519999999999999e-06 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351200+0000", 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.801e-06 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351205+0000", 2024-09-15T00:10:01.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379213+0000", 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028007539000000001 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379254+0000", 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1817999999999998e-05 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379310+0000", 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5850000000000002e-05 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4083 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086159193, 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028218838999999999, 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.491 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4083 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351214+0000", 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351216+0000", 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351222+0000", 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351226+0000", 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379333+0000", 2024-09-15T00:10:01.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028106740000000002 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379376+0000", 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3581000000000002e-05 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379432+0000", 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5868000000000001e-05 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4085 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086130689000001, 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311823, 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4085 2024-09-15T00:10:01.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0699999999999998e-07 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351244+0000", 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.782e-06 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351248+0000", 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6289999999999999e-06 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351250+0000", 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379457+0000", 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028206479 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379499+0000", 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1637000000000001e-05 2024-09-15T00:10:01.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379554+0000", 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5248999999999999e-05 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4086 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086118847999998, 2024-09-15T00:10:01.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02842368, 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4086 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351259+0000", 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9520000000000002e-06 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351262+0000", 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351265+0000", 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:10:01.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379576+0000", 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311228000000001 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379619+0000", 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2595000000000001e-05 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379678+0000", 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4087 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086104493000001, 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523903999999999, 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4087 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351269+0000", 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351270+0000", 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1090000000000001e-06 2024-09-15T00:10:01.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351273+0000", 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9189999999999999e-06 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351275+0000", 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379701+0000", 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028425813000000001 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379742+0000", 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0769000000000002e-05 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379792+0000", 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-15T00:10:01.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4088 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086088359000001, 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028621284, 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4088 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:10:01.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351285+0000", 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351286+0000", 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.186e-06 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351289+0000", 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351292+0000", 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379815+0000", 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523211 2024-09-15T00:10:01.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379857+0000", 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1807999999999997e-05 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379906+0000", 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8594999999999998e-05 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4090 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086077273999997, 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028748587999999999, 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4090 2024-09-15T00:10:01.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:10:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351296+0000", 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351297+0000", 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1060000000000001e-06 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351300+0000", 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9110000000000002e-06 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351302+0000", 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379947+0000", 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028645177000000001 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379991+0000", 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3680999999999997e-05 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380044+0000", 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2957000000000001e-05 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4091 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086067718999999, 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028864198000000001, 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4091 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351306+0000", 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351309+0000", 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351312+0000", 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380069+0000", 2024-09-15T00:10:01.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028757933999999999 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380113+0000", 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3810000000000002e-05 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380169+0000", 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5887999999999998e-05 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4092 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086058360999999, 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028973819000000001, 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4092 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:10:01.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351315+0000", 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9999999999999999e-07 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351316+0000", 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.95e-07 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351318+0000", 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8629999999999999e-06 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351321+0000", 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380195+0000", 2024-09-15T00:10:01.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028873868 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380238+0000", 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2883999999999998e-05 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380288+0000", 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0096000000000003e-05 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4094 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086048742000003, 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02907964, 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4094 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351325+0000", 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.054e-06 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351328+0000", 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351331+0000", 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4679999999999999e-06 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380312+0000", 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028981587999999999 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380355+0000", 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2688000000000002e-05 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380404+0000", 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8767999999999998e-05 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4095 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.086039354999997, 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029185068000000002, 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4095 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:10:01.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351334+0000", 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.04e-07 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351335+0000", 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0300000000000001e-06 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351339+0000", 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351344+0000", 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0560000000000004e-06 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380428+0000", 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02908444 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380470+0000", 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1338000000000003e-05 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380518+0000", 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8724999999999999e-05 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.538 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.539 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:01.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:01.541 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:01.538+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:01.542 INFO:teuthology.orchestra.run.smithi092.stdout:3903 read 0x2702a00 thru 0x2706d6e (0x436f bytes) 2024-09-15T00:10:01.542 INFO:teuthology.orchestra.run.smithi092.stdout:3904 punch from 0x2294400 to 0x22a0200, (0xbe00 bytes) 2024-09-15T00:10:01.542 INFO:teuthology.orchestra.run.smithi092.stdout:3907 write 0x3658400 thru 0x365e530 (0x6131 bytes) 2024-09-15T00:10:01.542 INFO:teuthology.orchestra.run.smithi092.stdout:3908 write 0xdab7e00 thru 0xdabe6d8 (0x68d9 bytes) 2024-09-15T00:10:01.542 INFO:teuthology.orchestra.run.smithi092.stdout:3910 read 0xf1e600 thru 0xf2d55e (0xef5f bytes) 2024-09-15T00:10:01.542 INFO:teuthology.orchestra.run.smithi092.stdout:3911 read 0xb08ea00 thru 0xb09221a (0x381b bytes) 2024-09-15T00:10:01.543 INFO:teuthology.orchestra.run.smithi092.stdout:3912 read 0x82c5a00 thru 0x82d1f0f (0xc510 bytes) 2024-09-15T00:10:01.543 INFO:teuthology.orchestra.run.smithi092.stdout:3914 punch from 0x72bbe00 to 0x72c3400, (0x7600 bytes) 2024-09-15T00:10:01.543 INFO:teuthology.orchestra.run.smithi092.stdout:3915 read 0xc36e000 thru 0xc377777 (0x9778 bytes) 2024-09-15T00:10:01.543 INFO:teuthology.orchestra.run.smithi092.stdout:3917 clone 28 order 25 su 16777216 sc 5 2024-09-15T00:10:01.640 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:01.639+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:01.685 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:01.685 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:01.685 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:01.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:01.741 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:01.740+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:01.837 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:01.837 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:01.837 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:01.837 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:01.837 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:01.842 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:01.841+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:01.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:01.943 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:01.942+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:01.989 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:01.989 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:01.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:01.989 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:01.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3427 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699514033, 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027724631, 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3427 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287582+0000", 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2599999999999998e-07 2024-09-15T00:10:01.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287584+0000", 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8759999999999999e-06 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287589+0000", 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287594+0000", 2024-09-15T00:10:01.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.357e-06 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315217+0000", 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027622785 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315257+0000", 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0228000000000002e-05 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315306+0000", 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9228999999999999e-05 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3428 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699496027999999, 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027928562000000001, 2024-09-15T00:10:01.994 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3428 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:10:01.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287600+0000", 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:01.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287602+0000", 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287608+0000", 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287613+0000", 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8740000000000003e-06 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315346+0000", 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027733099000000001 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315431+0000", 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4350000000000004e-05 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315528+0000", 2024-09-15T00:10:01.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7374000000000007e-05 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3429 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699476718, 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028028630999999998, 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:01.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3429 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:02.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287621+0000", 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287627+0000", 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8440000000000003e-06 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287631+0000", 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5009999999999998e-06 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315556+0000", 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027925241999999999 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315595+0000", 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8512e-05 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315647+0000", 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2318000000000003e-05 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.003 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3430 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699458644000003, 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028122114, 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3430 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:02.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287644+0000", 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4219999999999997e-06 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287650+0000", 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0920000000000003e-06 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287654+0000", 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4279999999999998e-06 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315671+0000", 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016592 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315709+0000", 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8231e-05 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315759+0000", 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9926999999999998e-05 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3431 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699435104000003, 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02831842, 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.008 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3431 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287661+0000", 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287663+0000", 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9570000000000001e-06 2024-09-15T00:10:02.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287668+0000", 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287673+0000", 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315799+0000", 2024-09-15T00:10:02.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028125741999999999 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315891+0000", 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2701000000000006e-05 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315979+0000", 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7287000000000006e-05 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3432 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699416720000002, 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028430624000000002, 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.013 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3432 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287681+0000", 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8530000000000001e-06 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287686+0000", 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4659999999999998e-06 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287691+0000", 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2230000000000001e-06 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316006+0000", 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028315344999999999 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316054+0000", 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8004999999999999e-05 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316109+0000", 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5374999999999998e-05 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3433 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699398963999997, 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028528593000000001, 2024-09-15T00:10:02.017 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3433 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:10:02.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287697+0000", 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287699+0000", 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8509999999999999e-06 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287704+0000", 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287710+0000", 2024-09-15T00:10:02.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316124+0000", 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028414466999999999 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316171+0000", 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6783999999999998e-05 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316225+0000", 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3891999999999998e-05 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3434 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699379579000002, 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028632801999999999, 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.022 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3434 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-07 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287718+0000", 2024-09-15T00:10:02.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287724+0000", 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8390000000000003e-06 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287729+0000", 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8389999999999997e-06 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316250+0000", 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028521135 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316295+0000", 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4540999999999999e-05 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316349+0000", 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4083999999999999e-05 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3435 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699348903000001, 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028723243999999998, 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.027 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3435 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287747+0000", 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4900000000000002e-07 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287748+0000", 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4190000000000001e-06 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287752+0000", 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7900000000000001e-06 2024-09-15T00:10:02.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287756+0000", 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1609999999999996e-06 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316375+0000", 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028618932 2024-09-15T00:10:02.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316417+0000", 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316470+0000", 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3436 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699333854999999, 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028827854, 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3436 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:10:02.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287766+0000", 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287771+0000", 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5140000000000001e-06 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287776+0000", 2024-09-15T00:10:02.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316494+0000", 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028718144000000001 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316538+0000", 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4129999999999999e-05 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316589+0000", 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1548999999999997e-05 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3437 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699315962999997, 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028972188999999999, 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3437 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287780+0000", 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287782+0000", 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8419999999999999e-06 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287787+0000", 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3059999999999997e-06 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287791+0000", 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9920000000000001e-06 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316615+0000", 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028824487999999999 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316668+0000", 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2883999999999997e-05 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316752+0000", 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3308999999999998e-05 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.040 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3438 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699298921, 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029124331999999999, 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3438 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287797+0000", 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:02.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287799+0000", 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287804+0000", 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7069999999999997e-06 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287808+0000", 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0720000000000001e-06 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316799+0000", 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028990241999999999 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316845+0000", 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5837000000000002e-05 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316921+0000", 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6255999999999996e-05 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3439 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699281382000002, 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029290665, 2024-09-15T00:10:02.045 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3439 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287816+0000", 2024-09-15T00:10:02.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6980000000000001e-06 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287824+0000", 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2500000000000006e-06 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287829+0000", 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316967+0000", 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029138319999999999 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317047+0000", 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0081999999999997e-05 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317105+0000", 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7497999999999997e-05 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3440 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699260322999997, 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029395451999999999, 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.050 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3440 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287837+0000", 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287843+0000", 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6799999999999998e-06 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287848+0000", 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317131+0000", 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029283337999999999 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317176+0000", 2024-09-15T00:10:02.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5176999999999998e-05 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317231+0000", 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4120999999999999e-05 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3441 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699242259000002, 2024-09-15T00:10:02.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029496321999999998, 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3441 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287855+0000", 2024-09-15T00:10:02.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7099999999999999e-06 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287861+0000", 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287866+0000", 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317252+0000", 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029386051999999999 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317297+0000", 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5164999999999999e-05 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317349+0000", 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2534000000000002e-05 2024-09-15T00:10:02.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3442 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699222992999999, 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029594697999999999, 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3442 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287873+0000", 2024-09-15T00:10:02.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-15T00:10:02.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287879+0000", 2024-09-15T00:10:02.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287885+0000", 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.755e-06 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287890+0000", 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317375+0000", 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029485457 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317416+0000", 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1217999999999997e-05 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317467+0000", 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0819999999999998e-05 2024-09-15T00:10:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3443 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699199653999997, 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029684676, 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3443 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287898+0000", 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287904+0000", 2024-09-15T00:10:02.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8860000000000001e-06 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287908+0000", 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5650000000000003e-06 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317488+0000", 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029579437 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317530+0000", 2024-09-15T00:10:02.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1900999999999997e-05 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317580+0000", 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0603000000000003e-05 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3444 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699174436, 2024-09-15T00:10:02.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029772823, 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3444 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:02.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287931+0000", 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0008e-05 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287938+0000", 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287943+0000", 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317603+0000", 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029659913999999999 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317646+0000", 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2367999999999998e-05 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317694+0000", 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8201999999999997e-05 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3445 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699145098000002, 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029863565000000002, 2024-09-15T00:10:02.073 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3445 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287951+0000", 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287953+0000", 2024-09-15T00:10:02.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.765e-06 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287958+0000", 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999996e-06 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287963+0000", 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.634e-06 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317714+0000", 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029751353000000001 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317755+0000", 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0342000000000001e-05 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317814+0000", 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9188999999999999e-05 2024-09-15T00:10:02.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3446 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.699126800000002, 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029967884, 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3446 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287971+0000", 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287977+0000", 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4330000000000003e-06 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287981+0000", 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3989999999999997e-06 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317837+0000", 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029855942 2024-09-15T00:10:02.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317878+0000", 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467000000000001e-05 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317936+0000", 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8100999999999998e-05 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.082 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:02.083 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:10:02.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:02.085 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:02.082+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:02.118 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone27 from 0xdabe800 (overlap 0xc93e00) to 0x649e00 2024-09-15T00:10:02.185 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:02.183+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:02.259 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:02.259 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:02.259 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:02.260 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:02.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:02.285 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:02.284+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:02.386 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:02.385+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:02.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:02.412 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:02.412 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:02.412 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:02.413 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:02.413 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:02.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:02.506 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:02.505+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:02.512 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:02.531 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:02.531 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:02.532 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-15T00:10:02.549 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:02.566 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:02.579 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:02.579 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:02.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:02.579 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:02.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.579 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.579 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.694112040999997, 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:02.584 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.694073410999998, 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.585 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:02.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:02.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:02.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.694047474000001, 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:02.590 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:02.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:02.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.594 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.694031406000001, 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.595 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:02.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:02.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:02.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:02.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:02.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.693890310999997, 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:02.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.604 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.693118556000002, 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:10:02.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:10:02.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.693033524999997, 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.609 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:02.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.692998508999999, 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:02.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.692972906000001, 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.618 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:02.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.692798639999999, 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:02.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.692567500999999, 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.627 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:02.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.691755557, 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.632 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.691675816999997, 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:02.636 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:02.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:02.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.691470791999997, 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:02.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.691259258999999, 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.645 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:02.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.688703252000003, 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:02.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.686720145999999, 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.654 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:02.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:02.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.686700575000003, 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:02.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11360 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.41597432899999998, 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045212219999999997, 2024-09-15T00:10:02.663 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11360 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.46e-07 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160944+0000", 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0420000000000001e-06 2024-09-15T00:10:02.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160949+0000", 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2320000000000001e-06 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160953+0000", 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0470000000000004e-06 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.205971+0000", 2024-09-15T00:10:02.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045017313000000003 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206046+0000", 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5457000000000006e-05 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206154+0000", 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010778299999999999 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.415957725, 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.668 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:02.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:02.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:02.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:02.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:02.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:02.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:02.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:02.672 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:02.673 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:02.674 INFO:teuthology.orchestra.run.smithi092.stdout:3921 read 0x5131a00 thru 0x5131a2f (0x30 bytes) 2024-09-15T00:10:02.674 INFO:teuthology.orchestra.run.smithi092.stdout:3922 trunc from 0xdabe800 to 0x10b400 2024-09-15T00:10:02.674 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:02.676 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:02.676 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:02.677 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:02.677 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:02.677 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:02.678 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:02.672+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:02.774 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:02.773+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:02.780 INFO:teuthology.orchestra.run.smithi092.stdout:3923 punch from 0x3ae00 to 0x41600, (0x6800 bytes) 2024-09-15T00:10:02.785 INFO:teuthology.orchestra.run.smithi092.stdout:3924 clone 29 order 20 su 524288 sc 2 2024-09-15T00:10:02.825 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:02.826 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:02.834 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:02.875 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:02.874+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:02.941 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:02.941 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:02.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:02.976 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:02.975+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:10:03.067 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:03.067 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:03.076 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:03.117 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:03.116+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:03.130 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone28 from 0x10b400 (overlap 0x10b400) to 0x0 2024-09-15T00:10:03.178 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:03.177+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:03.228 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:03.229 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:03.238 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:03.279 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:03.277+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:03.380 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:03.378+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:03.387 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:03.387 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:03.396 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:03.471 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-15T00:10:03.481 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:03.479+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:03.506 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:03.506 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:03.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:03.582 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:03.580+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:03.627 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:03.627 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:03.632 INFO:teuthology.orchestra.run.smithi092.stdout:3925 write 0x89b6e00 thru 0x89c2ccd (0xbece bytes) 2024-09-15T00:10:03.633 INFO:teuthology.orchestra.run.smithi092.stdout:3929 punch from 0x4881000 to 0x4890800, (0xf800 bytes) 2024-09-15T00:10:03.633 INFO:teuthology.orchestra.run.smithi092.stdout:3930 read 0x79f200 thru 0x7ab1b5 (0xbfb6 bytes) 2024-09-15T00:10:03.637 INFO:teuthology.orchestra.run.smithi092.stdout:3931 trunc from 0x89c2e00 to 0xc467800 2024-09-15T00:10:03.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:03.644 INFO:teuthology.orchestra.run.smithi092.stdout:3932 punch from 0xbce8200 to 0xbcecc00, (0x4a00 bytes) 2024-09-15T00:10:03.646 INFO:teuthology.orchestra.run.smithi092.stdout:3933 read 0x5072800 thru 0x50797bf (0x6fc0 bytes) 2024-09-15T00:10:03.646 INFO:teuthology.orchestra.run.smithi092.stdout:3936 read 0x7bf9c00 thru 0x7bff0b5 (0x54b6 bytes) 2024-09-15T00:10:03.653 INFO:teuthology.orchestra.run.smithi092.stdout:3937 trunc from 0xc467800 to 0x7374400 2024-09-15T00:10:03.667 INFO:teuthology.orchestra.run.smithi092.stdout:3940 trunc from 0x7374400 to 0x90f4a00 2024-09-15T00:10:03.674 INFO:teuthology.orchestra.run.smithi092.stdout:3946 read 0x7660e00 thru 0x76685fb (0x77fc bytes) 2024-09-15T00:10:03.676 INFO:teuthology.orchestra.run.smithi092.stdout:3947 punch from 0x1e73400 to 0x1e7ee00, (0xba00 bytes) 2024-09-15T00:10:03.676 INFO:teuthology.orchestra.run.smithi092.stdout:3948 clone 30 order 24 su 1048576 sc 9 2024-09-15T00:10:03.721 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:03.720+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:03.780 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:03.780 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:03.784 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:03.783+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:03.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:03.885 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:03.884+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:03.895 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-15T00:10:03.895 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:10:03.896 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T00:10:03.896 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 1.000000 2024-09-15T00:10:03.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-15T00:10:03.932 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:03.932 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:03.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:03.986 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:03.984+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:04.044 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:04.045 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:04.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:04.096 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:04.094+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:04.139 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone29 from 0x90f4a00 (overlap 0x10b400) to 0x85a00 2024-09-15T00:10:04.188 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:04.187+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:04.202 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:04.203 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:04.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:04.295 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:04.294+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:04.357 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:04.357 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:04.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:04.390 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:04.388+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-15T00:10:04.490 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:04.489+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:04.510 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:04.511 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:04.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:04.523 INFO:teuthology.orchestra.run.smithi092.stdout:3949 read 0x5983a00 thru 0x5985d36 (0x2337 bytes) 2024-09-15T00:10:04.529 INFO:teuthology.orchestra.run.smithi092.stdout:3950 trunc from 0x90f4a00 to 0xd5adc00 2024-09-15T00:10:04.540 INFO:teuthology.orchestra.run.smithi092.stdout:3951 write 0x97e9a00 thru 0x97f7fa4 (0xe5a5 bytes) 2024-09-15T00:10:04.541 INFO:teuthology.orchestra.run.smithi092.stdout:3953 punch from 0xa2e9600 to 0xa2f0200, (0x6c00 bytes) 2024-09-15T00:10:04.543 INFO:teuthology.orchestra.run.smithi092.stdout:3955 punch from 0x1c1b000 to 0x1c2a800, (0xf800 bytes) 2024-09-15T00:10:04.543 INFO:teuthology.orchestra.run.smithi092.stdout:3958 punch from 0x319a200 to 0x319b600, (0x1400 bytes) 2024-09-15T00:10:04.546 INFO:teuthology.orchestra.run.smithi092.stdout:3960 read 0xb785e00 thru 0xb786c7c (0xe7d bytes) 2024-09-15T00:10:04.546 INFO:teuthology.orchestra.run.smithi092.stdout:3962 write 0xa501c00 thru 0xa510bca (0xefcb bytes) 2024-09-15T00:10:04.546 INFO:teuthology.orchestra.run.smithi092.stdout:3966 write 0xbc5fc00 thru 0xbc62bcd (0x2fce bytes) 2024-09-15T00:10:04.546 INFO:teuthology.orchestra.run.smithi092.stdout:3967 read 0x5672400 thru 0x5676d23 (0x4924 bytes) 2024-09-15T00:10:04.546 INFO:teuthology.orchestra.run.smithi092.stdout:3969 write 0x1bf8c00 thru 0x1c01526 (0x8927 bytes) 2024-09-15T00:10:04.546 INFO:teuthology.orchestra.run.smithi092.stdout:3970 punch from 0xbb9c00 to 0xbc7c00, (0xe000 bytes) 2024-09-15T00:10:04.546 INFO:teuthology.orchestra.run.smithi092.stdout:3971 read 0xec9400 thru 0xed1bf6 (0x87f7 bytes) 2024-09-15T00:10:04.548 INFO:teuthology.orchestra.run.smithi092.stdout:3972 trunc from 0xd5adc00 to 0xa54be00 2024-09-15T00:10:04.571 INFO:teuthology.orchestra.run.smithi092.stdout:3974 trunc from 0xa54be00 to 0x6c11400 2024-09-15T00:10:04.592 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:04.591+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:04.596 INFO:teuthology.orchestra.run.smithi092.stdout:3977 write 0x62f7a00 thru 0x62fb817 (0x3e18 bytes) 2024-09-15T00:10:04.610 INFO:teuthology.orchestra.run.smithi092.stdout:3978 write 0xa1a4400 thru 0xa1a7877 (0x3478 bytes) 2024-09-15T00:10:04.617 INFO:teuthology.orchestra.run.smithi092.stdout:3981 read 0x491c000 thru 0x4921b1f (0x5b20 bytes) 2024-09-15T00:10:04.623 INFO:teuthology.orchestra.run.smithi092.stdout:3982 trunc from 0xa1a7a00 to 0x503fe00 2024-09-15T00:10:04.656 INFO:teuthology.orchestra.run.smithi092.stdout:3983 write 0x71f1200 thru 0x71fbc43 (0xaa44 bytes) 2024-09-15T00:10:04.660 INFO:teuthology.orchestra.run.smithi092.stdout:3986 read 0x2943e00 thru 0x294d26d (0x946e bytes) 2024-09-15T00:10:04.661 INFO:teuthology.orchestra.run.smithi092.stdout:3988 read 0x4d8a400 thru 0x4d94d25 (0xa926 bytes) 2024-09-15T00:10:04.661 INFO:teuthology.orchestra.run.smithi092.stdout:3989 punch from 0x505cc00 to 0x5066e00, (0xa200 bytes) 2024-09-15T00:10:04.661 INFO:teuthology.orchestra.run.smithi092.stdout:3995 punch from 0x1bae800 to 0x1bb1200, (0x2a00 bytes) 2024-09-15T00:10:04.662 INFO:teuthology.orchestra.run.smithi092.stdout:3996 write 0x3e3f800 thru 0x3e4b406 (0xbc07 bytes) 2024-09-15T00:10:04.663 INFO:teuthology.orchestra.run.smithi092.stdout:3997 write 0x5a43800 thru 0x5a48eef (0x56f0 bytes) 2024-09-15T00:10:04.664 INFO:teuthology.orchestra.run.smithi092.stdout:4002 read 0x2fb3e00 thru 0x2fb85d9 (0x47da bytes) 2024-09-15T00:10:04.665 INFO:teuthology.orchestra.run.smithi092.stdout:4004 write 0x6a8400 thru 0x6b6907 (0xe508 bytes) 2024-09-15T00:10:04.665 INFO:teuthology.orchestra.run.smithi092.stdout:4005 write 0x37c0800 thru 0x37c15b6 (0xdb7 bytes) 2024-09-15T00:10:04.665 INFO:teuthology.orchestra.run.smithi092.stdout:4009 write 0x667ce00 thru 0x6683cf8 (0x6ef9 bytes) 2024-09-15T00:10:04.666 INFO:teuthology.orchestra.run.smithi092.stdout:4010 write 0x5f9fe00 thru 0x5fa8974 (0x8b75 bytes) 2024-09-15T00:10:04.666 INFO:teuthology.orchestra.run.smithi092.stdout:4012 write 0x7ac800 thru 0x7ad9f7 (0x11f8 bytes) 2024-09-15T00:10:04.666 INFO:teuthology.orchestra.run.smithi092.stdout:4013 write 0x29bea00 thru 0x29cb783 (0xcd84 bytes) 2024-09-15T00:10:04.667 INFO:teuthology.orchestra.run.smithi092.stdout:4014 read 0x3d40a00 thru 0x3d45e4d (0x544e bytes) 2024-09-15T00:10:04.673 INFO:teuthology.orchestra.run.smithi092.stdout:4015 trunc from 0x71fbe00 to 0x2062200 2024-09-15T00:10:04.677 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:04.677 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:04.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:04.687 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:04.690 INFO:teuthology.orchestra.run.smithi092.stdout:4016 read 0x1f5000 thru 0x1f69fa (0x19fb bytes) 2024-09-15T00:10:04.698 INFO:teuthology.orchestra.run.smithi092.stdout:4018 read 0xe1e00 thru 0xf0046 (0xe247 bytes) 2024-09-15T00:10:04.699 INFO:teuthology.orchestra.run.smithi092.stdout:4019 read 0xeb2e00 thru 0xeb9d00 (0x6f01 bytes) 2024-09-15T00:10:04.700 INFO:teuthology.orchestra.run.smithi092.stdout:4022 write 0x1fd7400 thru 0x1fd86b1 (0x12b2 bytes) 2024-09-15T00:10:04.714 INFO:teuthology.orchestra.run.smithi092.stdout:4023 write 0x440ec00 thru 0x441529f (0x66a0 bytes) 2024-09-15T00:10:04.722 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:04.721+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:04.725 INFO:teuthology.orchestra.run.smithi092.stdout:4028 trunc from 0x4415400 to 0x6c9400 2024-09-15T00:10:04.767 INFO:teuthology.orchestra.run.smithi092.stdout:4029 write 0x2effc00 thru 0x2f06d7e (0x717f bytes) 2024-09-15T00:10:04.773 INFO:teuthology.orchestra.run.smithi092.stdout:4030 read 0x2954600 thru 0x2962cb9 (0xe6ba bytes) 2024-09-15T00:10:04.773 INFO:teuthology.orchestra.run.smithi092.stdout:4031 read 0x916200 thru 0x9184c4 (0x22c5 bytes) 2024-09-15T00:10:04.774 INFO:teuthology.orchestra.run.smithi092.stdout:4034 write 0x2bba000 thru 0x2bbd362 (0x3363 bytes) 2024-09-15T00:10:04.785 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:04.793 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:04.791+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:04.800 INFO:teuthology.orchestra.run.smithi092.stdout:4035 write 0xd901c00 thru 0xd90eefa (0xd2fb bytes) 2024-09-15T00:10:04.802 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:04.802 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:04.803 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:04.807 INFO:teuthology.orchestra.run.smithi092.stdout:4036 write 0x71ee400 thru 0x71f6e90 (0x8a91 bytes) 2024-09-15T00:10:04.807 INFO:teuthology.orchestra.run.smithi092.stdout:4037 read 0x3b08000 thru 0x3b1508e (0xd08f bytes) 2024-09-15T00:10:04.808 INFO:teuthology.orchestra.run.smithi092.stdout:4040 write 0x384c600 thru 0x3857ae2 (0xb4e3 bytes) 2024-09-15T00:10:04.808 INFO:teuthology.orchestra.run.smithi092.stdout:4042 write 0xb7ed800 thru 0xb7fcbae (0xf3af bytes) 2024-09-15T00:10:04.808 INFO:teuthology.orchestra.run.smithi092.stdout:4044 read 0x13f600 thru 0x14d311 (0xdd12 bytes) 2024-09-15T00:10:04.809 INFO:teuthology.orchestra.run.smithi092.stdout:4045 write 0x2cecc00 thru 0x2cf1881 (0x4c82 bytes) 2024-09-15T00:10:04.809 INFO:teuthology.orchestra.run.smithi092.stdout:4046 read 0x82a0400 thru 0x82ac49e (0xc09f bytes) 2024-09-15T00:10:04.809 INFO:teuthology.orchestra.run.smithi092.stdout:4048 read 0x4acda00 thru 0x4add130 (0xf731 bytes) 2024-09-15T00:10:04.810 INFO:teuthology.orchestra.run.smithi092.stdout:4050 write 0x63a8400 thru 0x63ae2b2 (0x5eb3 bytes) 2024-09-15T00:10:04.810 INFO:teuthology.orchestra.run.smithi092.stdout:4051 punch from 0xb114200 to 0xb117c00, (0x3a00 bytes) 2024-09-15T00:10:04.811 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:04.812 INFO:teuthology.orchestra.run.smithi092.stdout:4053 write 0xbc82e00 thru 0xbc87b0a (0x4d0b bytes) 2024-09-15T00:10:04.812 INFO:teuthology.orchestra.run.smithi092.stdout:4055 read 0x943a400 thru 0x9447333 (0xcf34 bytes) 2024-09-15T00:10:04.812 INFO:teuthology.orchestra.run.smithi092.stdout:4056 write 0xbef3800 thru 0xbef74ea (0x3ceb bytes) 2024-09-15T00:10:04.812 INFO:teuthology.orchestra.run.smithi092.stdout:4057 write 0x3e5c000 thru 0x3e5c3f4 (0x3f5 bytes) 2024-09-15T00:10:04.812 INFO:teuthology.orchestra.run.smithi092.stdout:4059 punch from 0x69e5000 to 0x69e9800, (0x4800 bytes) 2024-09-15T00:10:04.812 INFO:teuthology.orchestra.run.smithi092.stdout:4062 read 0x766a200 thru 0x7679f37 (0xfd38 bytes) 2024-09-15T00:10:04.813 INFO:teuthology.orchestra.run.smithi092.stdout:4064 punch from 0x89d7c00 to 0x89e3c00, (0xc000 bytes) 2024-09-15T00:10:04.813 INFO:teuthology.orchestra.run.smithi092.stdout:4065 read 0x7c7ec00 thru 0x7c8272a (0x3b2b bytes) 2024-09-15T00:10:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:04.823 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:04.828 INFO:teuthology.orchestra.run.smithi092.stdout:4066 trunc from 0xd90f000 to 0xdaec00 2024-09-15T00:10:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:04.858 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:04.871 INFO:teuthology.orchestra.run.smithi092.stdout:4067 write 0x9cb9e00 thru 0x9cc95e2 (0xf7e3 bytes) 2024-09-15T00:10:04.872 INFO:teuthology.orchestra.run.smithi092.stdout:4068 punch from 0x8549800 to 0x8554600, (0xae00 bytes) 2024-09-15T00:10:04.873 INFO:teuthology.orchestra.run.smithi092.stdout:4069 write 0x41a6000 thru 0x41ad345 (0x7346 bytes) 2024-09-15T00:10:04.876 INFO:teuthology.orchestra.run.smithi092.stdout:4070 read 0x243c400 thru 0x24499da (0xd5db bytes) 2024-09-15T00:10:04.878 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:04.878 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:04.881 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:04.881 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:04.882 INFO:teuthology.orchestra.run.smithi092.stdout:4071 trunc from 0x9cc9600 to 0x4d12600 2024-09-15T00:10:04.900 INFO:teuthology.orchestra.run.smithi092.stdout:4072 punch from 0x42c8c00 to 0x42cca00, (0x3e00 bytes) 2024-09-15T00:10:04.903 INFO:teuthology.orchestra.run.smithi092.stdout:4076 punch from 0x4521200 to 0x452a000, (0x8e00 bytes) 2024-09-15T00:10:04.903 INFO:teuthology.orchestra.run.smithi092.stdout:4077 read 0x1a6da00 thru 0x1a76958 (0x8f59 bytes) 2024-09-15T00:10:04.907 INFO:teuthology.orchestra.run.smithi092.stdout:4082 trunc from 0x4d12600 to 0x8536000 2024-09-15T00:10:04.918 INFO:teuthology.orchestra.run.smithi092.stdout:4087 trunc from 0x8536000 to 0x5edf400 2024-09-15T00:10:04.924 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:04.922+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:04.929 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:04.929 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:04.929 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:04.929 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:04.931 INFO:teuthology.orchestra.run.smithi092.stdout:4088 punch from 0x3c75e00 to 0x3c76400, (0x600 bytes) 2024-09-15T00:10:04.933 INFO:teuthology.orchestra.run.smithi092.stdout:4090 read 0x5a0200 thru 0x5afcd2 (0xfad3 bytes) 2024-09-15T00:10:04.936 INFO:teuthology.orchestra.run.smithi092.stdout:4091 read 0x3bc400 thru 0x3c4e30 (0x8a31 bytes) 2024-09-15T00:10:04.936 INFO:teuthology.orchestra.run.smithi092.stdout:4092 punch from 0x3bea00 to 0x3c5c00, (0x7200 bytes) 2024-09-15T00:10:04.936 INFO:teuthology.orchestra.run.smithi092.stdout:4093 read 0x4985a00 thru 0x4995454 (0xfa55 bytes) 2024-09-15T00:10:04.937 INFO:teuthology.orchestra.run.smithi092.stdout:4094 write 0x51ec00 thru 0x52c5ce (0xd9cf bytes) 2024-09-15T00:10:04.938 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:04.956 INFO:teuthology.orchestra.run.smithi092.stdout:4095 write 0xe74ac00 thru 0xe755ab0 (0xaeb1 bytes) 2024-09-15T00:10:04.964 INFO:teuthology.orchestra.run.smithi092.stdout:4096 trunc from 0xe755c00 to 0x7a8d400 2024-09-15T00:10:04.995 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:04.993+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:04.997 INFO:teuthology.orchestra.run.smithi092.stdout:4098 write 0xea0c800 thru 0xea1ad6d (0xe56e bytes) 2024-09-15T00:10:05.014 INFO:teuthology.orchestra.run.smithi092.stdout:4099 trunc from 0xea1ae00 to 0xc92c00 2024-09-15T00:10:05.041 INFO:teuthology.orchestra.run.smithi092.stdout:4100 read 0x38f200 thru 0x393e0b (0x4c0c bytes) 2024-09-15T00:10:05.057 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-15T00:10:05.058 INFO:teuthology.orchestra.run.smithi092.stdout:4102 trunc from 0xc92c00 to 0xdbd2e00 2024-09-15T00:10:05.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:10:05.073 INFO:teuthology.orchestra.run.smithi092.stdout:4103 read 0xbd55e00 thru 0xbd59318 (0x3519 bytes) 2024-09-15T00:10:05.077 INFO:teuthology.orchestra.run.smithi092.stdout:4110 trunc from 0xdbd2e00 to 0xa8cac00 2024-09-15T00:10:05.085 INFO:teuthology.orchestra.run.smithi092.stdout:4112 write 0x1403a00 thru 0x1404838 (0xe39 bytes) 2024-09-15T00:10:05.087 INFO:teuthology.orchestra.run.smithi092.stdout:4115 write 0x49d6e00 thru 0x49e2ae0 (0xbce1 bytes) 2024-09-15T00:10:05.087 INFO:teuthology.orchestra.run.smithi092.stdout:4119 punch from 0x375dc00 to 0x375ea00, (0xe00 bytes) 2024-09-15T00:10:05.090 INFO:teuthology.orchestra.run.smithi092.stdout:4120 trunc from 0xa8cac00 to 0xb5fae00 2024-09-15T00:10:05.093 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:05.093 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:05.093 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:05.093 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:05.094 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:05.096 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:05.095+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:05.097 INFO:teuthology.orchestra.run.smithi092.stdout:4121 write 0x1bd9c00 thru 0x1be3ed4 (0xa2d5 bytes) 2024-09-15T00:10:05.099 INFO:teuthology.orchestra.run.smithi092.stdout:4122 read 0x2d62400 thru 0x2d6a1b8 (0x7db9 bytes) 2024-09-15T00:10:05.100 INFO:teuthology.orchestra.run.smithi092.stdout:4124 punch from 0xaa93800 to 0xaa94600, (0xe00 bytes) 2024-09-15T00:10:05.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:05.111 INFO:teuthology.orchestra.run.smithi092.stdout:4125 write 0xdaa0e00 thru 0xdaaff9f (0xf1a0 bytes) 2024-09-15T00:10:05.112 INFO:teuthology.orchestra.run.smithi092.stdout:4131 read 0x844fc00 thru 0x84520b2 (0x24b3 bytes) 2024-09-15T00:10:05.113 INFO:teuthology.orchestra.run.smithi092.stdout:4132 write 0x15c9e00 thru 0x15d5112 (0xb313 bytes) 2024-09-15T00:10:05.113 INFO:teuthology.orchestra.run.smithi092.stdout:4134 write 0xa400000 thru 0xa40aaeb (0xaaec bytes) 2024-09-15T00:10:05.113 INFO:teuthology.orchestra.run.smithi092.stdout:4136 read 0x9972800 thru 0x99814c0 (0xecc1 bytes) 2024-09-15T00:10:05.124 INFO:teuthology.orchestra.run.smithi092.stdout:4140 trunc from 0xdab0000 to 0x4e7be00 2024-09-15T00:10:05.148 INFO:teuthology.orchestra.run.smithi092.stdout:4142 trunc from 0x4e7be00 to 0x7674a00 2024-09-15T00:10:05.157 INFO:teuthology.orchestra.run.smithi092.stdout:4143 punch from 0x32dfc00 to 0x32e5000, (0x5400 bytes) 2024-09-15T00:10:05.181 INFO:teuthology.orchestra.run.smithi092.stdout:4144 write 0xc770c00 thru 0xc7756d1 (0x4ad2 bytes) 2024-09-15T00:10:05.194 INFO:teuthology.orchestra.run.smithi092.stdout:4148 write 0xc921800 thru 0xc92d39c (0xbb9d bytes) 2024-09-15T00:10:05.195 INFO:teuthology.orchestra.run.smithi092.stdout:4149 read 0x50c5200 thru 0x50ca542 (0x5343 bytes) 2024-09-15T00:10:05.197 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:05.195+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:05.199 INFO:teuthology.orchestra.run.smithi092.stdout:4150 punch from 0x138d200 to 0x1396200, (0x9000 bytes) 2024-09-15T00:10:05.199 INFO:teuthology.orchestra.run.smithi092.stdout:4153 write 0x6f19a00 thru 0x6f1dbc3 (0x41c4 bytes) 2024-09-15T00:10:05.205 INFO:teuthology.orchestra.run.smithi092.stdout:4154 trunc from 0xc92d400 to 0x80b5800 2024-09-15T00:10:05.260 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:05.260 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230469530000001, 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:10:05.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.265 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230449274999998, 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.266 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.270 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230429528999998, 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:05.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:05.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:05.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:05.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:05.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:05.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:05.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:05.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:05.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:05.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:05.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:05.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:05.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:05.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:05.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:05.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.277 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230409579000003, 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.278 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.282 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230382079000002, 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:05.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:05.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230365237999997, 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.287 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:05.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.291 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230349240000002, 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.292 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:05.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.296 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230326251999998, 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:05.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230300368000002, 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.302 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:05.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230283941000003, 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:05.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:05.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230269669999998, 2024-09-15T00:10:05.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:05.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:05.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.316 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.230260352999998, 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:05.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:05.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.228591340999998, 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:05.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.228556277999999, 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.326 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:05.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:05.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:05.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:05.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:05.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:05.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:05.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.228531251, 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.334 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:05.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:05.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.338 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.228506942999999, 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.339 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.340 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:05.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.340 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:05.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2674 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3768320~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.226360597000003, 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059850591000000002, 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2674 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031671+0000", 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031682+0000", 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1248e-05 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031693+0000", 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1395e-05 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031705+0000", 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2106000000000001e-05 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091352+0000", 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059646421999999998 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091441+0000", 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9174999999999999e-05 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091521+0000", 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9636999999999997e-05 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.226314692000003, 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:10:05.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:10:05.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:10:05.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.226290560000002, 2024-09-15T00:10:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:05.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:05.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.226261188000002, 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:05.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.362 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.363 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:05.364 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:05.365 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:05.362+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:05.423 INFO:teuthology.orchestra.run.smithi092.stdout:4155 read 0x7bca00 thru 0x7c97c4 (0xcdc5 bytes) 2024-09-15T00:10:05.424 INFO:teuthology.orchestra.run.smithi092.stdout:4156 read 0x33200 thru 0x3c54c (0x934d bytes) 2024-09-15T00:10:05.424 INFO:teuthology.orchestra.run.smithi092.stdout:4159 read 0x6fb9000 thru 0x6fbca1a (0x3a1b bytes) 2024-09-15T00:10:05.424 INFO:teuthology.orchestra.run.smithi092.stdout:4160 punch from 0x609d200 to 0x60a7600, (0xa400 bytes) 2024-09-15T00:10:05.424 INFO:teuthology.orchestra.run.smithi092.stdout:4161 read 0x4df4e00 thru 0x4dfb696 (0x6897 bytes) 2024-09-15T00:10:05.425 INFO:teuthology.orchestra.run.smithi092.stdout:4162 write 0x2e5e00 thru 0x2ec0bf (0x62c0 bytes) 2024-09-15T00:10:05.437 INFO:teuthology.orchestra.run.smithi092.stdout:4163 write 0x939ac00 thru 0x93a7e4d (0xd24e bytes) 2024-09-15T00:10:05.438 INFO:teuthology.orchestra.run.smithi092.stdout:4164 write 0x2b5cc00 thru 0x2b600e3 (0x34e4 bytes) 2024-09-15T00:10:05.439 INFO:teuthology.orchestra.run.smithi092.stdout:4165 read 0x6da000 thru 0x6e38d0 (0x98d1 bytes) 2024-09-15T00:10:05.444 INFO:teuthology.orchestra.run.smithi092.stdout:4166 trunc from 0x93a8000 to 0xd01c000 2024-09-15T00:10:05.455 INFO:teuthology.orchestra.run.smithi092.stdout:4169 read 0x6a81200 thru 0x6a86fa3 (0x5da4 bytes) 2024-09-15T00:10:05.461 INFO:teuthology.orchestra.run.smithi092.stdout:4171 trunc from 0xd01c000 to 0x8e31c00 2024-09-15T00:10:05.465 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:05.464+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:05.472 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:05.472 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:05.472 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:05.472 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:05.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:05.482 INFO:teuthology.orchestra.run.smithi092.stdout:4174 punch from 0x246f200 to 0x2471000, (0x1e00 bytes) 2024-09-15T00:10:05.482 INFO:teuthology.orchestra.run.smithi092.stdout:4175 write 0x71a800 thru 0x7250d8 (0xa8d9 bytes) 2024-09-15T00:10:05.490 INFO:teuthology.orchestra.run.smithi092.stdout:4176 trunc from 0x8e31c00 to 0x12b0600 2024-09-15T00:10:05.525 INFO:teuthology.orchestra.run.smithi092.stdout:4177 trunc from 0x12b0600 to 0x84ea200 2024-09-15T00:10:05.536 INFO:teuthology.orchestra.run.smithi092.stdout:4178 trunc from 0x84ea200 to 0x7256000 2024-09-15T00:10:05.547 INFO:teuthology.orchestra.run.smithi092.stdout:4179 trunc from 0x7256000 to 0x9861200 2024-09-15T00:10:05.556 INFO:teuthology.orchestra.run.smithi092.stdout:4180 write 0x2b35800 thru 0x2b3b4ff (0x5d00 bytes) 2024-09-15T00:10:05.557 INFO:teuthology.orchestra.run.smithi092.stdout:4184 write 0x9577000 thru 0x9581ab3 (0xaab4 bytes) 2024-09-15T00:10:05.565 INFO:teuthology.orchestra.run.smithi092.stdout:4188 trunc from 0x9861200 to 0x2bf3800 2024-09-15T00:10:05.566 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:05.564+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:05.587 INFO:teuthology.orchestra.run.smithi092.stdout:4189 punch from 0xf7ac00 to 0xf7ec00, (0x4000 bytes) 2024-09-15T00:10:05.593 INFO:teuthology.orchestra.run.smithi092.stdout:4192 write 0x325200 thru 0x330626 (0xb427 bytes) 2024-09-15T00:10:05.593 INFO:teuthology.orchestra.run.smithi092.stdout:4194 read 0x6e9600 thru 0x6ed916 (0x4317 bytes) 2024-09-15T00:10:05.594 INFO:teuthology.orchestra.run.smithi092.stdout:4195 punch from 0x13d8600 to 0x13e3c00, (0xb600 bytes) 2024-09-15T00:10:05.603 INFO:teuthology.orchestra.run.smithi092.stdout:4198 trunc from 0x2bf3800 to 0xa1de600 2024-09-15T00:10:05.612 INFO:teuthology.orchestra.run.smithi092.stdout:4199 read 0x910aa00 thru 0x91141b3 (0x97b4 bytes) 2024-09-15T00:10:05.618 INFO:teuthology.orchestra.run.smithi092.stdout:4204 read 0x714cc00 thru 0x715b54f (0xe950 bytes) 2024-09-15T00:10:05.619 INFO:teuthology.orchestra.run.smithi092.stdout:4208 write 0x83ec000 thru 0x83f9002 (0xd003 bytes) 2024-09-15T00:10:05.619 INFO:teuthology.orchestra.run.smithi092.stdout:4209 read 0x7711400 thru 0x7714bd2 (0x37d3 bytes) 2024-09-15T00:10:05.619 INFO:teuthology.orchestra.run.smithi092.stdout:4211 read 0x42a7200 thru 0x42aeb7a (0x797b bytes) 2024-09-15T00:10:05.633 INFO:teuthology.orchestra.run.smithi092.stdout:4212 write 0xd34fe00 thru 0xd3500b2 (0x2b3 bytes) 2024-09-15T00:10:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:4214 read 0xb9dc800 thru 0xb9e3bd7 (0x73d8 bytes) 2024-09-15T00:10:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:4215 read 0x9135c00 thru 0x9138f11 (0x3312 bytes) 2024-09-15T00:10:05.634 INFO:teuthology.orchestra.run.smithi092.stdout:4217 punch from 0x9ddfe00 to 0x9de9c00, (0x9e00 bytes) 2024-09-15T00:10:05.635 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:05.635 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:05.635 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:05.635 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:05.635 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:05.636 INFO:teuthology.orchestra.run.smithi092.stdout:4218 trunc from 0xd350200 to 0xc3fc800 2024-09-15T00:10:05.643 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:05.648 INFO:teuthology.orchestra.run.smithi092.stdout:4219 write 0x5f2a600 thru 0x5f351de (0xabdf bytes) 2024-09-15T00:10:05.663 INFO:teuthology.orchestra.run.smithi092.stdout:4220 write 0xeddd000 thru 0xede088f (0x3890 bytes) 2024-09-15T00:10:05.664 INFO:teuthology.orchestra.run.smithi092.stdout:4221 write 0xb702600 thru 0xb70b63c (0x903d bytes) 2024-09-15T00:10:05.665 INFO:teuthology.orchestra.run.smithi092.stdout:4222 read 0xcfce000 thru 0xcfdd2a3 (0xf2a4 bytes) 2024-09-15T00:10:05.665 INFO:teuthology.orchestra.run.smithi092.stdout:4224 write 0xdd06800 thru 0xdd0dee8 (0x76e9 bytes) 2024-09-15T00:10:05.665 INFO:teuthology.orchestra.run.smithi092.stdout:4228 read 0x9025200 thru 0x90284fe (0x32ff bytes) 2024-09-15T00:10:05.665 INFO:teuthology.orchestra.run.smithi092.stdout:4229 read 0x3e2b400 thru 0x3e2e32a (0x2f2b bytes) 2024-09-15T00:10:05.665 INFO:teuthology.orchestra.run.smithi092.stdout:4231 punch from 0xbbbf600 to 0xbbca200, (0xac00 bytes) 2024-09-15T00:10:05.666 INFO:teuthology.orchestra.run.smithi092.stdout:4232 write 0xc7b2600 thru 0xc7bd656 (0xb057 bytes) 2024-09-15T00:10:05.666 INFO:teuthology.orchestra.run.smithi092.stdout:4233 write 0xc39cc00 thru 0xc3a046a (0x386b bytes) 2024-09-15T00:10:05.666 INFO:teuthology.orchestra.run.smithi092.stdout:4234 read 0x5e24a00 thru 0x5e2ca07 (0x8008 bytes) 2024-09-15T00:10:05.676 INFO:teuthology.orchestra.run.smithi092.stdout:4236 trunc from 0xede0a00 to 0x5a56e00 2024-09-15T00:10:05.687 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:05.686+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:05.699 INFO:teuthology.orchestra.run.smithi092.stdout:4240 trunc from 0x5a56e00 to 0x5200e00 2024-09-15T00:10:05.755 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:05.756 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:05.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:05.756 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:05.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.756 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.756 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:05.757 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.726798115999998, 2024-09-15T00:10:05.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:05.757 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.757 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:05.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.758 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:05.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:05.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:05.762 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.762 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.762 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.762 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.762 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:05.762 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.726776198000003, 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:05.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:05.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:05.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.726754925999998, 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:05.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:05.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:05.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.725013666000002, 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:05.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:05.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:05.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:05.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:05.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.778 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.724991760000002, 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:05.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:05.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:05.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.724969668, 2024-09-15T00:10:05.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:05.787 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.724937928999999, 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.788 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:05.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.724639725999999, 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:05.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.724606021, 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:05.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.802 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.720303948000002, 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.803 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:05.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.719979078000001, 2024-09-15T00:10:05.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:05.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:05.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.719568672999998, 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.812 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:05.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.816 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.719513769999999, 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.817 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:05.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.719481735000002, 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:05.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.718003905000003, 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:05.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.717982190000001, 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:05.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:05.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.717960711000003, 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:05.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:05.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:05.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:05.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:05.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.841 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.841 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.841 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.841 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:05.841 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.715947606999997, 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:05.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:05.845 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.715927966000002, 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:05.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:05.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.715881418000002, 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:05.851 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:05.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:05.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:05.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:05.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:05.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.856 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:05.856 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:05.856 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:05.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:05.858 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:05.856+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:05.922 INFO:teuthology.orchestra.run.smithi092.stdout:4241 trunc from 0x5200e00 to 0x97d3600 2024-09-15T00:10:05.930 INFO:teuthology.orchestra.run.smithi092.stdout:4242 write 0x7f06200 thru 0x7f0dbe1 (0x79e2 bytes) 2024-09-15T00:10:05.945 INFO:teuthology.orchestra.run.smithi092.stdout:4248 write 0xb36c400 thru 0xb370fff (0x4c00 bytes) 2024-09-15T00:10:05.946 INFO:teuthology.orchestra.run.smithi092.stdout:4249 write 0x69dd400 thru 0x69e4633 (0x7234 bytes) 2024-09-15T00:10:05.948 INFO:teuthology.orchestra.run.smithi092.stdout:4250 punch from 0x8ead600 to 0x8eb6800, (0x9200 bytes) 2024-09-15T00:10:05.948 INFO:teuthology.orchestra.run.smithi092.stdout:4251 punch from 0x1a0a600 to 0x1a13e00, (0x9800 bytes) 2024-09-15T00:10:05.948 INFO:teuthology.orchestra.run.smithi092.stdout:4255 punch from 0x6452200 to 0x6459800, (0x7600 bytes) 2024-09-15T00:10:05.949 INFO:teuthology.orchestra.run.smithi092.stdout:4257 read 0xafaac00 thru 0xafb74cf (0xc8d0 bytes) 2024-09-15T00:10:05.949 INFO:teuthology.orchestra.run.smithi092.stdout:4258 read 0x8f62600 thru 0x8f636fa (0x10fb bytes) 2024-09-15T00:10:05.949 INFO:teuthology.orchestra.run.smithi092.stdout:4259 read 0x5664a00 thru 0x5664eeb (0x4ec bytes) 2024-09-15T00:10:05.954 INFO:teuthology.orchestra.run.smithi092.stdout:4260 trunc from 0xb371000 to 0xec1e200 2024-09-15T00:10:05.958 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:05.956+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:05.964 INFO:teuthology.orchestra.run.smithi092.stdout:4261 write 0x61ffe00 thru 0x620c7ba (0xc9bb bytes) 2024-09-15T00:10:05.970 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:05.970 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:05.970 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:05.970 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:05.971 INFO:teuthology.orchestra.run.smithi092.stdout:4264 trunc from 0xec1e200 to 0x9310000 2024-09-15T00:10:05.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:05.988 INFO:teuthology.orchestra.run.smithi092.stdout:4268 punch from 0x5b55600 to 0x5b57a00, (0x2400 bytes) 2024-09-15T00:10:05.991 INFO:teuthology.orchestra.run.smithi092.stdout:4270 read 0x5708800 thru 0x570bc57 (0x3458 bytes) 2024-09-15T00:10:06.002 INFO:teuthology.orchestra.run.smithi092.stdout:4271 trunc from 0x9310000 to 0x9ea400 2024-09-15T00:10:06.029 INFO:teuthology.orchestra.run.smithi092.stdout:4275 read 0x1ed800 thru 0x1fcab3 (0xf2b4 bytes) 2024-09-15T00:10:06.030 INFO:teuthology.orchestra.run.smithi092.stdout:4277 read 0x594800 thru 0x59505f (0x860 bytes) 2024-09-15T00:10:06.030 INFO:teuthology.orchestra.run.smithi092.stdout:4278 read 0x7e1e00 thru 0x7ef38b (0xd58c bytes) 2024-09-15T00:10:06.031 INFO:teuthology.orchestra.run.smithi092.stdout:4281 read 0x3d0c00 thru 0x3de4e6 (0xd8e7 bytes) 2024-09-15T00:10:06.031 INFO:teuthology.orchestra.run.smithi092.stdout:4282 read 0x605800 thru 0x606b19 (0x131a bytes) 2024-09-15T00:10:06.032 INFO:teuthology.orchestra.run.smithi092.stdout:4284 read 0x349200 thru 0x350c52 (0x7a53 bytes) 2024-09-15T00:10:06.032 INFO:teuthology.orchestra.run.smithi092.stdout:4292 read 0x73d000 thru 0x73e11a (0x111b bytes) 2024-09-15T00:10:06.057 INFO:teuthology.orchestra.run.smithi092.stdout:4295 write 0xd615a00 thru 0xd61ed18 (0x9319 bytes) 2024-09-15T00:10:06.057 INFO:teuthology.orchestra.run.smithi092.stdout:4296 read 0x52cb000 thru 0x52cb535 (0x536 bytes) 2024-09-15T00:10:06.057 INFO:teuthology.orchestra.run.smithi092.stdout:4297 read 0x9585a00 thru 0x958798d (0x1f8e bytes) 2024-09-15T00:10:06.057 INFO:teuthology.orchestra.run.smithi092.stdout:4298 read 0xa8fb000 thru 0xa905f16 (0xaf17 bytes) 2024-09-15T00:10:06.058 INFO:teuthology.orchestra.run.smithi092.stdout:4299 read 0x4a05a00 thru 0x4a107c6 (0xadc7 bytes) 2024-09-15T00:10:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:10:06.065 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:06.063+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:06.071 INFO:teuthology.orchestra.run.smithi092.stdout:4300 trunc from 0xd61ee00 to 0x1c32600 2024-09-15T00:10:06.109 INFO:teuthology.orchestra.run.smithi092.stdout:4301 read 0xde4200 thru 0xde8d8d (0x4b8e bytes) 2024-09-15T00:10:06.117 INFO:teuthology.orchestra.run.smithi092.stdout:4307 trunc from 0x1c32600 to 0x87e3e00 2024-09-15T00:10:06.126 INFO:teuthology.orchestra.run.smithi092.stdout:4308 punch from 0x11cf800 to 0x11db000, (0xb800 bytes) 2024-09-15T00:10:06.130 INFO:teuthology.orchestra.run.smithi092.stdout:4310 trunc from 0x87e3e00 to 0x6da7400 2024-09-15T00:10:06.132 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:06.132 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:06.132 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:06.133 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:06.133 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:06.141 INFO:teuthology.orchestra.run.smithi092.stdout:4311 read 0x6218000 thru 0x622352d (0xb52e bytes) 2024-09-15T00:10:06.142 INFO:teuthology.orchestra.run.smithi092.stdout:4312 punch from 0x121dc00 to 0x1221200, (0x3600 bytes) 2024-09-15T00:10:06.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:06.144 INFO:teuthology.orchestra.run.smithi092.stdout:4313 trunc from 0x6da7400 to 0x6ef2e00 2024-09-15T00:10:06.157 INFO:teuthology.orchestra.run.smithi092.stdout:4316 trunc from 0x6ef2e00 to 0x8fe7000 2024-09-15T00:10:06.174 INFO:teuthology.orchestra.run.smithi092.stdout:4317 trunc from 0x8fe7000 to 0x64f1400 2024-09-15T00:10:06.185 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:06.183+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:06.206 INFO:teuthology.orchestra.run.smithi092.stdout:4318 write 0x968e400 thru 0x969d0ef (0xecf0 bytes) 2024-09-15T00:10:06.221 INFO:teuthology.orchestra.run.smithi092.stdout:4322 trunc from 0x969d200 to 0x107a200 2024-09-15T00:10:06.251 INFO:teuthology.orchestra.run.smithi092.stdout:4325 read 0x1072a00 thru 0x107a04f (0x7650 bytes) 2024-09-15T00:10:06.253 INFO:teuthology.orchestra.run.smithi092.stdout:4327 read 0xc72c00 thru 0xc787d8 (0x5bd9 bytes) 2024-09-15T00:10:06.253 INFO:teuthology.orchestra.run.smithi092.stdout:4329 read 0xbbfc00 thru 0xbc9e47 (0xa248 bytes) 2024-09-15T00:10:06.259 INFO:teuthology.orchestra.run.smithi092.stdout:4330 trunc from 0x107a200 to 0x509e200 2024-09-15T00:10:06.261 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:06.260+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:06.269 INFO:teuthology.orchestra.run.smithi092.stdout:4334 read 0x40b8200 thru 0x40be7c0 (0x65c1 bytes) 2024-09-15T00:10:06.269 INFO:teuthology.orchestra.run.smithi092.stdout:4337 write 0x4899000 thru 0x489fdad (0x6dae bytes) 2024-09-15T00:10:06.270 INFO:teuthology.orchestra.run.smithi092.stdout:4339 read 0xbf3600 thru 0xbf6ee3 (0x38e4 bytes) 2024-09-15T00:10:06.271 INFO:teuthology.orchestra.run.smithi092.stdout:4341 read 0x3708200 thru 0x3713424 (0xb225 bytes) 2024-09-15T00:10:06.271 INFO:teuthology.orchestra.run.smithi092.stdout:4342 write 0x20e0400 thru 0x20ee117 (0xdd18 bytes) 2024-09-15T00:10:06.296 INFO:teuthology.orchestra.run.smithi092.stdout:4344 write 0x929a400 thru 0x92a1b97 (0x7798 bytes) 2024-09-15T00:10:06.296 INFO:teuthology.orchestra.run.smithi092.stdout:4345 read 0x39ad000 thru 0x39ae9fe (0x19ff bytes) 2024-09-15T00:10:06.296 INFO:teuthology.orchestra.run.smithi092.stdout:4347 read 0x28c6200 thru 0x28cd68e (0x748f bytes) 2024-09-15T00:10:06.297 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:06.297 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:06.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:06.297 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:06.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.298 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4070 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.298 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:10:06.298 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944354635000003, 2024-09-15T00:10:06.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02702953, 2024-09-15T00:10:06.298 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.298 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.298 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4070 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350999+0000", 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351003+0000", 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351007+0000", 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6820000000000001e-06 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351010+0000", 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7269999999999999e-06 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377929+0000", 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026919887999999999 2024-09-15T00:10:06.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377969+0000", 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9909e-05 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378028+0000", 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8578000000000001e-05 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4071 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:10:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944339755000001, 2024-09-15T00:10:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027136008, 2024-09-15T00:10:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4071 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351014+0000", 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351015+0000", 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7039999999999999e-06 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351020+0000", 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351023+0000", 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378053+0000", 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027029610999999999 2024-09-15T00:10:06.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378094+0000", 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1826999999999998e-05 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378149+0000", 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4815000000000003e-05 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4072 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944315076000002, 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027223404999999999, 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4072 2024-09-15T00:10:06.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351040+0000", 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351046+0000", 2024-09-15T00:10:06.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351051+0000", 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378171+0000", 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027120590999999999 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378209+0000", 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7308999999999998e-05 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378261+0000", 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2753999999999997e-05 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4074 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944296180999999, 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027334509999999999, 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4074 2024-09-15T00:10:06.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9200000000000002e-07 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351059+0000", 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.012e-06 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351065+0000", 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2129999999999996e-06 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351070+0000", 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8470000000000003e-06 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378285+0000", 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027214449000000002 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378334+0000", 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378391+0000", 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7046000000000002e-05 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.317 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4075 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944277309999997, 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027546923000000001, 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.318 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4075 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351078+0000", 2024-09-15T00:10:06.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.905e-06 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351085+0000", 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351090+0000", 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378423+0000", 2024-09-15T00:10:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027333051000000001 2024-09-15T00:10:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378523+0000", 2024-09-15T00:10:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9959999999999998e-05 2024-09-15T00:10:06.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378623+0000", 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9962000000000003e-05 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4077 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944254626000003, 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027660619000000001, 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4077 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351099+0000", 2024-09-15T00:10:06.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351101+0000", 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351107+0000", 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7980000000000002e-06 2024-09-15T00:10:06.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351111+0000", 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378657+0000", 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027545463999999999 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378702+0000", 2024-09-15T00:10:06.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5809999999999997e-05 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378759+0000", 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4078 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:10:06.328 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944226626000003, 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027808405000000001, 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4078 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351127+0000", 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351129+0000", 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351135+0000", 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3820000000000001e-06 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351141+0000", 2024-09-15T00:10:06.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2259999999999996e-06 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378785+0000", 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027644592999999999 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378855+0000", 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0276000000000003e-05 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378935+0000", 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9361000000000001e-05 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4079 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944204988999999, 2024-09-15T00:10:06.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027913778, 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4079 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:06.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351150+0000", 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9010000000000001e-06 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351157+0000", 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351162+0000", 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378958+0000", 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027795786999999999 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379004+0000", 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6124000000000001e-05 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379062+0000", 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7630999999999998e-05 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4081 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944181307999997, 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016658, 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4081 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351174+0000", 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-15T00:10:06.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351181+0000", 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9070000000000001e-06 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351186+0000", 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.651e-06 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379085+0000", 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027898757999999999 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379130+0000", 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5126000000000001e-05 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379188+0000", 2024-09-15T00:10:06.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8548999999999998e-05 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4082 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944161254000001, 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028118457999999999, 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4082 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:06.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351194+0000", 2024-09-15T00:10:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7519999999999999e-06 2024-09-15T00:10:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351200+0000", 2024-09-15T00:10:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.801e-06 2024-09-15T00:10:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351205+0000", 2024-09-15T00:10:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:10:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379213+0000", 2024-09-15T00:10:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028007539000000001 2024-09-15T00:10:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379254+0000", 2024-09-15T00:10:06.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1817999999999998e-05 2024-09-15T00:10:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379310+0000", 2024-09-15T00:10:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5850000000000002e-05 2024-09-15T00:10:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4083 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:10:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944139534000001, 2024-09-15T00:10:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028218838999999999, 2024-09-15T00:10:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.350 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4083 2024-09-15T00:10:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:10:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:10:06.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351214+0000", 2024-09-15T00:10:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351216+0000", 2024-09-15T00:10:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-15T00:10:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351222+0000", 2024-09-15T00:10:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-15T00:10:06.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351226+0000", 2024-09-15T00:10:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:10:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379333+0000", 2024-09-15T00:10:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028106740000000002 2024-09-15T00:10:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379376+0000", 2024-09-15T00:10:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3581000000000002e-05 2024-09-15T00:10:06.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379432+0000", 2024-09-15T00:10:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5868000000000001e-05 2024-09-15T00:10:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4085 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944111030000002, 2024-09-15T00:10:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311823, 2024-09-15T00:10:06.358 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4085 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0699999999999998e-07 2024-09-15T00:10:06.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351244+0000", 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.782e-06 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351248+0000", 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6289999999999999e-06 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351250+0000", 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379457+0000", 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028206479 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379499+0000", 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1637000000000001e-05 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379554+0000", 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5248999999999999e-05 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4086 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944099188999999, 2024-09-15T00:10:06.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02842368, 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4086 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:06.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351259+0000", 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9520000000000002e-06 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351262+0000", 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351265+0000", 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:10:06.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379576+0000", 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311228000000001 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379619+0000", 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2595000000000001e-05 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379678+0000", 2024-09-15T00:10:06.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4087 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944084834000002, 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523903999999999, 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4087 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:10:06.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351269+0000", 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351270+0000", 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1090000000000001e-06 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351273+0000", 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9189999999999999e-06 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351275+0000", 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379701+0000", 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028425813000000001 2024-09-15T00:10:06.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379742+0000", 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0769000000000002e-05 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379792+0000", 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4088 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944068700000003, 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028621284, 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4088 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351285+0000", 2024-09-15T00:10:06.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351286+0000", 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.186e-06 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351289+0000", 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351292+0000", 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379815+0000", 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523211 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379857+0000", 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1807999999999997e-05 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379906+0000", 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8594999999999998e-05 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4090 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944057614999998, 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028748587999999999, 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4090 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351296+0000", 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351297+0000", 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1060000000000001e-06 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351300+0000", 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9110000000000002e-06 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351302+0000", 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379947+0000", 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028645177000000001 2024-09-15T00:10:06.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379991+0000", 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3680999999999997e-05 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380044+0000", 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2957000000000001e-05 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4091 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.94404806, 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028864198000000001, 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4091 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.384 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351306+0000", 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351309+0000", 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-15T00:10:06.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351312+0000", 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380069+0000", 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028757933999999999 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380113+0000", 2024-09-15T00:10:06.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3810000000000002e-05 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380169+0000", 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5887999999999998e-05 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4092 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:10:06.388 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944038702, 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028973819000000001, 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4092 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:10:06.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351315+0000", 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9999999999999999e-07 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351316+0000", 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.95e-07 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351318+0000", 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8629999999999999e-06 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351321+0000", 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380195+0000", 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028873868 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380238+0000", 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2883999999999998e-05 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380288+0000", 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0096000000000003e-05 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4094 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944029082999997, 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02907964, 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4094 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351325+0000", 2024-09-15T00:10:06.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.054e-06 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351328+0000", 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351331+0000", 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4679999999999999e-06 2024-09-15T00:10:06.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380312+0000", 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028981587999999999 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380355+0000", 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2688000000000002e-05 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380404+0000", 2024-09-15T00:10:06.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8767999999999998e-05 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4095 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.944019695999998, 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029185068000000002, 2024-09-15T00:10:06.398 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4095 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351334+0000", 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.04e-07 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351335+0000", 2024-09-15T00:10:06.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0300000000000001e-06 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351339+0000", 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351344+0000", 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0560000000000004e-06 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380428+0000", 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02908444 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380470+0000", 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1338000000000003e-05 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380518+0000", 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8724999999999999e-05 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.403 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.403 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:06.404 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:06.403+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:06.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4348 trunc from 0x92a1c00 to 0xb3cf400 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4351 trunc from 0xb3cf400 to 0xd26ae00 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4352 write 0x7b4ca00 thru 0x7b57315 (0xa916 bytes) 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4353 punch from 0xa0d3200 to 0xa0dda00, (0xa800 bytes) 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4355 read 0xac2b800 thru 0xac2fdce (0x45cf bytes) 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4358 read 0x43b2e00 thru 0x43bd645 (0xa846 bytes) 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4359 write 0x92c0a00 thru 0x92c8b8a (0x818b bytes) 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4360 read 0xad2f800 thru 0xad3a794 (0xaf95 bytes) 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4365 punch from 0x9cec000 to 0x9ced200, (0x1200 bytes) 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4366 write 0x65b9200 thru 0x65c75c1 (0xe3c2 bytes) 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4368 write 0xece4200 thru 0xecf1916 (0xd717 bytes) 2024-09-15T00:10:06.407 INFO:teuthology.orchestra.run.smithi092.stdout:4369 punch from 0xb572600 to 0xb578200, (0x5c00 bytes) 2024-09-15T00:10:06.408 INFO:teuthology.orchestra.run.smithi092.stdout:4370 punch from 0xb5b2c00 to 0xb5b7e00, (0x5200 bytes) 2024-09-15T00:10:06.408 INFO:teuthology.orchestra.run.smithi092.stdout:4372 write 0xc91ce00 thru 0xc92249e (0x569f bytes) 2024-09-15T00:10:06.408 INFO:teuthology.orchestra.run.smithi092.stdout:4374 read 0x1d59c00 thru 0x1d6401c (0xa41d bytes) 2024-09-15T00:10:06.408 INFO:teuthology.orchestra.run.smithi092.stdout:4375 punch from 0x940c000 to 0x9414c00, (0x8c00 bytes) 2024-09-15T00:10:06.408 INFO:teuthology.orchestra.run.smithi092.stdout:4376 read 0xd530400 thru 0xd534e99 (0x4a9a bytes) 2024-09-15T00:10:06.408 INFO:teuthology.orchestra.run.smithi092.stdout:4378 trunc from 0xecf1a00 to 0xb1eb800 2024-09-15T00:10:06.408 INFO:teuthology.orchestra.run.smithi092.stdout:4379 punch from 0x2d4f800 to 0x2d5b000, (0xb800 bytes) 2024-09-15T00:10:06.408 INFO:teuthology.orchestra.run.smithi092.stdout:4380 trunc from 0xb1eb800 to 0x4d1ba00 2024-09-15T00:10:06.408 INFO:teuthology.orchestra.run.smithi092.stdout:4381 read 0x2e5a000 thru 0x2e60c89 (0x6c8a bytes) 2024-09-15T00:10:06.415 INFO:teuthology.orchestra.run.smithi092.stdout:4383 write 0xcae8800 thru 0xcaf6d01 (0xe502 bytes) 2024-09-15T00:10:06.431 INFO:teuthology.orchestra.run.smithi092.stdout:4384 write 0xe3b7400 thru 0xe3c2351 (0xaf52 bytes) 2024-09-15T00:10:06.432 INFO:teuthology.orchestra.run.smithi092.stdout:4386 read 0x6876000 thru 0x68828fa (0xc8fb bytes) 2024-09-15T00:10:06.447 INFO:teuthology.orchestra.run.smithi092.stdout:4390 trunc from 0xe3c2400 to 0x4e92400 2024-09-15T00:10:06.467 INFO:teuthology.orchestra.run.smithi092.stdout:4391 read 0x4331200 thru 0x433a84d (0x964e bytes) 2024-09-15T00:10:06.468 INFO:teuthology.orchestra.run.smithi092.stdout:4392 read 0x3601200 thru 0x3608a64 (0x7865 bytes) 2024-09-15T00:10:06.469 INFO:teuthology.orchestra.run.smithi092.stdout:4393 read 0xeb3c00 thru 0xebe833 (0xac34 bytes) 2024-09-15T00:10:06.469 INFO:teuthology.orchestra.run.smithi092.stdout:4396 read 0x38f6200 thru 0x39005c1 (0xa3c2 bytes) 2024-09-15T00:10:06.472 INFO:teuthology.orchestra.run.smithi092.stdout:4397 write 0x15d0c00 thru 0x15d50e9 (0x44ea bytes) 2024-09-15T00:10:06.477 INFO:teuthology.orchestra.run.smithi092.stdout:4399 trunc from 0x4e92400 to 0x649a00 2024-09-15T00:10:06.501 INFO:teuthology.orchestra.run.smithi092.stdout:4400 read 0x7fe00 thru 0x82541 (0x2742 bytes) 2024-09-15T00:10:06.503 INFO:teuthology.orchestra.run.smithi092.stdout:4403 read 0x22c400 thru 0x2394e7 (0xd0e8 bytes) 2024-09-15T00:10:06.504 INFO:teuthology.orchestra.run.smithi092.stdout:4404 read 0x59c200 thru 0x5a1f7a (0x5d7b bytes) 2024-09-15T00:10:06.504 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:06.503+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:06.504 INFO:teuthology.orchestra.run.smithi092.stdout:4405 trunc from 0x649a00 to 0xa45a00 2024-09-15T00:10:06.524 INFO:teuthology.orchestra.run.smithi092.stdout:4406 trunc from 0xa45a00 to 0x94fb400 2024-09-15T00:10:06.525 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:06.525 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:15647 3.3 3.43237d83 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:06.524821+0000", 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.6732e-05, 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2438000000000002e-05, 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15647 2024-09-15T00:10:06.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:06.524821+0000", 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:06.524821+0000", 2024-09-15T00:10:06.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:06.524822+0000", 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7599999999999997e-07 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:06.524825+0000", 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2940000000000001e-06 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:06.524833+0000", 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4279999999999996e-06 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:06.524838+0000", 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9810000000000003e-06 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:06.524874+0000", 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5991999999999997e-05 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.530 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:10:06.530 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-15T00:10:06.530 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:06.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:06.546 INFO:teuthology.orchestra.run.smithi092.stdout:4408 write 0xc024200 thru 0xc02552c (0x132d bytes) 2024-09-15T00:10:06.546 INFO:teuthology.orchestra.run.smithi092.stdout:4411 punch from 0xabf4a00 to 0xabf7c00, (0x3200 bytes) 2024-09-15T00:10:06.547 INFO:teuthology.orchestra.run.smithi092.stdout:4412 read 0xbd32a00 thru 0xbd338e2 (0xee3 bytes) 2024-09-15T00:10:06.548 INFO:teuthology.orchestra.run.smithi092.stdout:4413 read 0xadf1600 thru 0xadfd2e5 (0xbce6 bytes) 2024-09-15T00:10:06.548 INFO:teuthology.orchestra.run.smithi092.stdout:4424 read 0xb30f000 thru 0xb31eda3 (0xfda4 bytes) 2024-09-15T00:10:06.558 INFO:teuthology.orchestra.run.smithi092.stdout:4425 trunc from 0xc025600 to 0x2fb4600 2024-09-15T00:10:06.601 INFO:teuthology.orchestra.run.smithi092.stdout:4426 write 0xeb3d800 thru 0xeb3ed7a (0x157b bytes) 2024-09-15T00:10:06.602 INFO:teuthology.orchestra.run.smithi092.stdout:4427 write 0x8b94800 thru 0x8b98345 (0x3b46 bytes) 2024-09-15T00:10:06.602 INFO:teuthology.orchestra.run.smithi092.stdout:4428 write 0xb7e5000 thru 0xb7eb191 (0x6192 bytes) 2024-09-15T00:10:06.605 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:06.604+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:06.612 INFO:teuthology.orchestra.run.smithi092.stdout:4429 trunc from 0xeb3ee00 to 0x603ce00 2024-09-15T00:10:06.650 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:06.650 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:06.650 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:06.650 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:06.651 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:06.653 INFO:teuthology.orchestra.run.smithi092.stdout:4431 write 0x9c82600 thru 0x9c82932 (0x333 bytes) 2024-09-15T00:10:06.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:06.660 INFO:teuthology.orchestra.run.smithi092.stdout:4432 read 0x900dc00 thru 0x900f5a2 (0x19a3 bytes) 2024-09-15T00:10:06.660 INFO:teuthology.orchestra.run.smithi092.stdout:4433 read 0x4585c00 thru 0x458d2b5 (0x76b6 bytes) 2024-09-15T00:10:06.674 INFO:teuthology.orchestra.run.smithi092.stdout:4434 write 0xc492e00 thru 0xc498e2b (0x602c bytes) 2024-09-15T00:10:06.681 INFO:teuthology.orchestra.run.smithi092.stdout:4435 write 0x9ca7e00 thru 0x9caa1a9 (0x23aa bytes) 2024-09-15T00:10:06.681 INFO:teuthology.orchestra.run.smithi092.stdout:4437 punch from 0x3675e00 to 0x3683a00, (0xdc00 bytes) 2024-09-15T00:10:06.683 INFO:teuthology.orchestra.run.smithi092.stdout:4438 read 0x1ac1400 thru 0x1ac82b0 (0x6eb1 bytes) 2024-09-15T00:10:06.683 INFO:teuthology.orchestra.run.smithi092.stdout:4439 write 0xb567c00 thru 0xb5763d1 (0xe7d2 bytes) 2024-09-15T00:10:06.684 INFO:teuthology.orchestra.run.smithi092.stdout:4440 write 0x89fd800 thru 0x8a00cf5 (0x34f6 bytes) 2024-09-15T00:10:06.684 INFO:teuthology.orchestra.run.smithi092.stdout:4441 punch from 0x8783e00 to 0x8787c00, (0x3e00 bytes) 2024-09-15T00:10:06.684 INFO:teuthology.orchestra.run.smithi092.stdout:4442 read 0x2ce7c00 thru 0x2cea020 (0x2421 bytes) 2024-09-15T00:10:06.684 INFO:teuthology.orchestra.run.smithi092.stdout:4443 read 0x6340c00 thru 0x634caf5 (0xbef6 bytes) 2024-09-15T00:10:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:4444 write 0x8aeba00 thru 0x8aef89d (0x3e9e bytes) 2024-09-15T00:10:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:4445 write 0x19a0a00 thru 0x19a0e6b (0x46c bytes) 2024-09-15T00:10:06.685 INFO:teuthology.orchestra.run.smithi092.stdout:4452 read 0xa5b5c00 thru 0xa5bf7fc (0x9bfd bytes) 2024-09-15T00:10:06.686 INFO:teuthology.orchestra.run.smithi092.stdout:4453 punch from 0x9f82a00 to 0x9f88400, (0x5a00 bytes) 2024-09-15T00:10:06.686 INFO:teuthology.orchestra.run.smithi092.stdout:4456 read 0x7c41600 thru 0x7c4b2dd (0x9cde bytes) 2024-09-15T00:10:06.686 INFO:teuthology.orchestra.run.smithi092.stdout:4457 read 0x49d400 thru 0x4aaf15 (0xdb16 bytes) 2024-09-15T00:10:06.687 INFO:teuthology.orchestra.run.smithi092.stdout:4458 read 0xce3e00 thru 0xcf0b82 (0xcd83 bytes) 2024-09-15T00:10:06.689 INFO:teuthology.orchestra.run.smithi092.stdout:4460 punch from 0x785e400 to 0x786b600, (0xd200 bytes) 2024-09-15T00:10:06.689 INFO:teuthology.orchestra.run.smithi092.stdout:4461 write 0x7d7e00 thru 0x7e7a61 (0xfc62 bytes) 2024-09-15T00:10:06.692 INFO:teuthology.orchestra.run.smithi092.stdout:4462 read 0xa146400 thru 0xa151187 (0xad88 bytes) 2024-09-15T00:10:06.702 INFO:teuthology.orchestra.run.smithi092.stdout:4463 write 0xcd86a00 thru 0xcd8ac96 (0x4297 bytes) 2024-09-15T00:10:06.706 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:06.704+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:06.709 INFO:teuthology.orchestra.run.smithi092.stdout:4464 punch from 0x4e42800 to 0x4e42c00, (0x400 bytes) 2024-09-15T00:10:06.710 INFO:teuthology.orchestra.run.smithi092.stdout:4465 write 0x5546a00 thru 0x55478e9 (0xeea bytes) 2024-09-15T00:10:06.711 INFO:teuthology.orchestra.run.smithi092.stdout:4467 clone 31 order 18 su 65536 sc 12 2024-09-15T00:10:06.803 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3427 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513354270000001, 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027724631, 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3427 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287582+0000", 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2599999999999998e-07 2024-09-15T00:10:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287584+0000", 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8759999999999999e-06 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287589+0000", 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287594+0000", 2024-09-15T00:10:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.357e-06 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315217+0000", 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027622785 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315257+0000", 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0228000000000002e-05 2024-09-15T00:10:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315306+0000", 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9228999999999999e-05 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3428 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513336265, 2024-09-15T00:10:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027928562000000001, 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3428 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287600+0000", 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287602+0000", 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287608+0000", 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287613+0000", 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8740000000000003e-06 2024-09-15T00:10:06.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315346+0000", 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027733099000000001 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315431+0000", 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4350000000000004e-05 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315528+0000", 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7374000000000007e-05 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3429 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513316955000001, 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028028630999999998, 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.813 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3429 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287621+0000", 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287627+0000", 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8440000000000003e-06 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287631+0000", 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5009999999999998e-06 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315556+0000", 2024-09-15T00:10:06.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027925241999999999 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315595+0000", 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8512e-05 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315647+0000", 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2318000000000003e-05 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3430 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513298880999997, 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028122114, 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3430 2024-09-15T00:10:06.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287644+0000", 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4219999999999997e-06 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287650+0000", 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0920000000000003e-06 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287654+0000", 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4279999999999998e-06 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315671+0000", 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016592 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315709+0000", 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8231e-05 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315759+0000", 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9926999999999998e-05 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3431 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:10:06.822 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513275341000003, 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02831842, 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3431 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287661+0000", 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-15T00:10:06.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287663+0000", 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9570000000000001e-06 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287668+0000", 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287673+0000", 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315799+0000", 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028125741999999999 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315891+0000", 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2701000000000006e-05 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315979+0000", 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7287000000000006e-05 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3432 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513256957000003, 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028430624000000002, 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3432 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287681+0000", 2024-09-15T00:10:06.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8530000000000001e-06 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287686+0000", 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4659999999999998e-06 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287691+0000", 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2230000000000001e-06 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316006+0000", 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028315344999999999 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316054+0000", 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8004999999999999e-05 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316109+0000", 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5374999999999998e-05 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3433 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513239200999998, 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028528593000000001, 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3433 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287697+0000", 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287699+0000", 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8509999999999999e-06 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287704+0000", 2024-09-15T00:10:06.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287710+0000", 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316124+0000", 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028414466999999999 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316171+0000", 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6783999999999998e-05 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316225+0000", 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3891999999999998e-05 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3434 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.836 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513219816000003, 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028632801999999999, 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3434 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-07 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287718+0000", 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287724+0000", 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8390000000000003e-06 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287729+0000", 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8389999999999997e-06 2024-09-15T00:10:06.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316250+0000", 2024-09-15T00:10:06.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028521135 2024-09-15T00:10:06.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316295+0000", 2024-09-15T00:10:06.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4540999999999999e-05 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316349+0000", 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4083999999999999e-05 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3435 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:10:06.841 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513189140000001, 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028723243999999998, 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3435 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:10:06.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287747+0000", 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4900000000000002e-07 2024-09-15T00:10:06.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287748+0000", 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4190000000000001e-06 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287752+0000", 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7900000000000001e-06 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287756+0000", 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1609999999999996e-06 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316375+0000", 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028618932 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316417+0000", 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316470+0000", 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3436 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513174092, 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028827854, 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3436 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287766+0000", 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:10:06.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287771+0000", 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5140000000000001e-06 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287776+0000", 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316494+0000", 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028718144000000001 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316538+0000", 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4129999999999999e-05 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316589+0000", 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1548999999999997e-05 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3437 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513156199999997, 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028972188999999999, 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.851 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3437 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287780+0000", 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287782+0000", 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8419999999999999e-06 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287787+0000", 2024-09-15T00:10:06.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3059999999999997e-06 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287791+0000", 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9920000000000001e-06 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316615+0000", 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028824487999999999 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316668+0000", 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2883999999999997e-05 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316752+0000", 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3308999999999998e-05 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3438 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:10:06.855 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513139158000001, 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029124331999999999, 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3438 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287797+0000", 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287799+0000", 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287804+0000", 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7069999999999997e-06 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287808+0000", 2024-09-15T00:10:06.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0720000000000001e-06 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316799+0000", 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028990241999999999 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316845+0000", 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5837000000000002e-05 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316921+0000", 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6255999999999996e-05 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3439 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513121619000003, 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029290665, 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3439 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287816+0000", 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6980000000000001e-06 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287824+0000", 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2500000000000006e-06 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287829+0000", 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316967+0000", 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029138319999999999 2024-09-15T00:10:06.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317047+0000", 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0081999999999997e-05 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317105+0000", 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7497999999999997e-05 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3440 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513100559999998, 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029395451999999999, 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3440 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287837+0000", 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287843+0000", 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6799999999999998e-06 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287848+0000", 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317131+0000", 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029283337999999999 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317176+0000", 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5176999999999998e-05 2024-09-15T00:10:06.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317231+0000", 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4120999999999999e-05 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3441 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513082496000003, 2024-09-15T00:10:06.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029496321999999998, 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3441 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:06.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287855+0000", 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7099999999999999e-06 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287861+0000", 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287866+0000", 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317252+0000", 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029386051999999999 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317297+0000", 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5164999999999999e-05 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317349+0000", 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2534000000000002e-05 2024-09-15T00:10:06.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3442 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.51306323, 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029594697999999999, 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3442 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287873+0000", 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287879+0000", 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287885+0000", 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.755e-06 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287890+0000", 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317375+0000", 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029485457 2024-09-15T00:10:06.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317416+0000", 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1217999999999997e-05 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317467+0000", 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0819999999999998e-05 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3443 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513039890999998, 2024-09-15T00:10:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029684676, 2024-09-15T00:10:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.879 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3443 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287898+0000", 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:06.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287904+0000", 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8860000000000001e-06 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287908+0000", 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5650000000000003e-06 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317488+0000", 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029579437 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317530+0000", 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1900999999999997e-05 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317580+0000", 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0603000000000003e-05 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.883 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3444 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.513014673000001, 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029772823, 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3444 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287931+0000", 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0008e-05 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287938+0000", 2024-09-15T00:10:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287943+0000", 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317603+0000", 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029659913999999999 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317646+0000", 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2367999999999998e-05 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317694+0000", 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8201999999999997e-05 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.888 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3445 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.512985335000003, 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029863565000000002, 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3445 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287951+0000", 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:10:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287953+0000", 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.765e-06 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287958+0000", 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999996e-06 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287963+0000", 2024-09-15T00:10:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.634e-06 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317714+0000", 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029751353000000001 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317755+0000", 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0342000000000001e-05 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317814+0000", 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9188999999999999e-05 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3446 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.512967037000003, 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029967884, 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:06.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3446 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287971+0000", 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287977+0000", 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4330000000000003e-06 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287981+0000", 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3989999999999997e-06 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317837+0000", 2024-09-15T00:10:06.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029855942 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317878+0000", 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467000000000001e-05 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317936+0000", 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8100999999999998e-05 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:06.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:06.898 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:06.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:06.899 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:06.897+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:06.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:06.999 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:06.997+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:07.002 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:07.002 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:07.002 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:07.002 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:07.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:07.023 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:07.023 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:07.040 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:07.057 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:07.057 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:07.075 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:07.075 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:07.093 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:07.093 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:07.096 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:07.096 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:07.121 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:07.122 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:07.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:07.137 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:07.135+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:07.173 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone30 from 0xcd8ae00 (overlap 0x649a00) to 0x3c5c00 2024-09-15T00:10:07.201 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:07.200+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:07.243 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:07.243 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:07.252 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:07.302 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:07.300+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:07.363 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:07.363 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:07.371 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:07.403 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:07.401+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:07.481 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:07.481 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:07.491 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:07.534 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:07.532+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:07.600 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:07.600 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:07.605 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:07.603+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:07.608 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:07.620 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-15T00:10:07.683 INFO:teuthology.orchestra.run.smithi092.stdout:4470 read 0x14ad200 thru 0x14b48aa (0x76ab bytes) 2024-09-15T00:10:07.687 INFO:teuthology.orchestra.run.smithi092.stdout:4476 punch from 0xbe63e00 to 0xbe70000, (0xc200 bytes) 2024-09-15T00:10:07.695 INFO:teuthology.orchestra.run.smithi092.stdout:4477 read 0xcb34e00 thru 0xcb3a250 (0x5451 bytes) 2024-09-15T00:10:07.695 INFO:teuthology.orchestra.run.smithi092.stdout:4478 write 0x3635c00 thru 0x364260f (0xca10 bytes) 2024-09-15T00:10:07.696 INFO:teuthology.orchestra.run.smithi092.stdout:4480 read 0xa7d8e00 thru 0xa7e6afd (0xdcfe bytes) 2024-09-15T00:10:07.696 INFO:teuthology.orchestra.run.smithi092.stdout:4481 write 0xb807400 thru 0xb809f3b (0x2b3c bytes) 2024-09-15T00:10:07.696 INFO:teuthology.orchestra.run.smithi092.stdout:4482 read 0x8282600 thru 0x828713a (0x4b3b bytes) 2024-09-15T00:10:07.696 INFO:teuthology.orchestra.run.smithi092.stdout:4486 punch from 0x70ec00 to 0x711400, (0x2800 bytes) 2024-09-15T00:10:07.706 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:07.704+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:07.710 INFO:teuthology.orchestra.run.smithi092.stdout:4490 write 0x9496800 thru 0x949c31c (0x5b1d bytes) 2024-09-15T00:10:07.710 INFO:teuthology.orchestra.run.smithi092.stdout:4491 read 0x11f6a00 thru 0x11fc596 (0x5b97 bytes) 2024-09-15T00:10:07.710 INFO:teuthology.orchestra.run.smithi092.stdout:4492 write 0x80bce00 thru 0x80c80f9 (0xb2fa bytes) 2024-09-15T00:10:07.710 INFO:teuthology.orchestra.run.smithi092.stdout:4493 write 0x3a09c00 thru 0x3a19b61 (0xff62 bytes) 2024-09-15T00:10:07.724 INFO:teuthology.orchestra.run.smithi092.stdout:4495 trunc from 0xcd8ae00 to 0x1e41800 2024-09-15T00:10:07.753 INFO:teuthology.orchestra.run.smithi092.stdout:4500 read 0x109c000 thru 0x10a685c (0xa85d bytes) 2024-09-15T00:10:07.755 INFO:teuthology.orchestra.run.smithi092.stdout:4502 trunc from 0x1e41800 to 0xf98600 2024-09-15T00:10:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:07.762 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:07.772 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:07.803 INFO:teuthology.orchestra.run.smithi092.stdout:4503 write 0x63b1a00 thru 0x63bcc35 (0xb236 bytes) 2024-09-15T00:10:07.807 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:07.805+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:07.809 INFO:teuthology.orchestra.run.smithi092.stdout:4505 read 0x34e3e00 thru 0x34e4e55 (0x1056 bytes) 2024-09-15T00:10:07.809 INFO:teuthology.orchestra.run.smithi092.stdout:4509 write 0x4d54e00 thru 0x4d5a3b2 (0x55b3 bytes) 2024-09-15T00:10:07.810 INFO:teuthology.orchestra.run.smithi092.stdout:4510 punch from 0x66d800 to 0x671a00, (0x4200 bytes) 2024-09-15T00:10:07.818 INFO:teuthology.orchestra.run.smithi092.stdout:4513 trunc from 0x63bce00 to 0x4b6400 2024-09-15T00:10:07.868 INFO:teuthology.orchestra.run.smithi092.stdout:4515 write 0xa731e00 thru 0xa738227 (0x6428 bytes) 2024-09-15T00:10:07.868 INFO:teuthology.orchestra.run.smithi092.stdout:4517 read 0x7def400 thru 0x7df7163 (0x7d64 bytes) 2024-09-15T00:10:07.869 INFO:teuthology.orchestra.run.smithi092.stdout:4519 read 0x5a01c00 thru 0x5a01c69 (0x6a bytes) 2024-09-15T00:10:07.869 INFO:teuthology.orchestra.run.smithi092.stdout:4520 write 0x9998600 thru 0x999c4dd (0x3ede bytes) 2024-09-15T00:10:07.869 INFO:teuthology.orchestra.run.smithi092.stdout:4521 read 0x1300400 thru 0x130f793 (0xf394 bytes) 2024-09-15T00:10:07.883 INFO:teuthology.orchestra.run.smithi092.stdout:4522 write 0xcdf2e00 thru 0xcdfc799 (0x999a bytes) 2024-09-15T00:10:07.883 INFO:teuthology.orchestra.run.smithi092.stdout:4523 read 0x62e2200 thru 0x62eee3a (0xcc3b bytes) 2024-09-15T00:10:07.888 INFO:teuthology.orchestra.run.smithi092.stdout:4524 trunc from 0xcdfc800 to 0x8b1fc00 2024-09-15T00:10:07.904 INFO:teuthology.orchestra.run.smithi092.stdout:4525 clone 32 order 20 su 4096 sc 4 2024-09-15T00:10:07.908 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:07.906+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:07.923 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:07.923 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:07.932 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:08.016 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:08.014+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:08.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:10:08.081 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:08.081 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:08.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:08.135 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:08.134+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:08.183 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone31 from 0x8b1fc00 (overlap 0x4b6400) to 0x34c600 2024-09-15T00:10:08.205 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:08.206 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:08.210 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:08.208+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:08.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:08.311 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:08.309+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:08.362 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:08.363 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:08.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:08.411 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:08.409+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:08.512 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:08.510+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:08.517 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:08.518 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:08.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:08.613 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:08.612+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:08.631 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:08.631 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:08.636 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-15T00:10:08.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:08.710 INFO:teuthology.orchestra.run.smithi092.stdout:4526 write 0x88eda00 thru 0x88eee57 (0x1458 bytes) 2024-09-15T00:10:08.711 INFO:teuthology.orchestra.run.smithi092.stdout:4527 write 0x856e800 thru 0x857d059 (0xe85a bytes) 2024-09-15T00:10:08.715 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:08.713+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:08.735 INFO:teuthology.orchestra.run.smithi092.stdout:4530 write 0xa72aa00 thru 0xa73492c (0x9f2d bytes) 2024-09-15T00:10:08.741 INFO:teuthology.orchestra.run.smithi092.stdout:4531 write 0x1b83400 thru 0x1b8c0b5 (0x8cb6 bytes) 2024-09-15T00:10:08.762 INFO:teuthology.orchestra.run.smithi092.stdout:4533 write 0xdae8800 thru 0xdaee6a5 (0x5ea6 bytes) 2024-09-15T00:10:08.768 INFO:teuthology.orchestra.run.smithi092.stdout:4534 read 0xc8a3600 thru 0xc8b21c1 (0xebc2 bytes) 2024-09-15T00:10:08.769 INFO:teuthology.orchestra.run.smithi092.stdout:4537 punch from 0x5b34c00 to 0x5b35c00, (0x1000 bytes) 2024-09-15T00:10:08.770 INFO:teuthology.orchestra.run.smithi092.stdout:4539 punch from 0xa98ac00 to 0xa98ea00, (0x3e00 bytes) 2024-09-15T00:10:08.770 INFO:teuthology.orchestra.run.smithi092.stdout:4541 punch from 0x642be00 to 0x6434000, (0x8200 bytes) 2024-09-15T00:10:08.773 INFO:teuthology.orchestra.run.smithi092.stdout:4542 write 0x179de00 thru 0x17a681a (0x8a1b bytes) 2024-09-15T00:10:08.773 INFO:teuthology.orchestra.run.smithi092.stdout:4546 trunc from 0xdaee800 to 0xd0c2400 2024-09-15T00:10:08.800 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:08.800 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:08.801 INFO:teuthology.orchestra.run.smithi092.stdout:4547 punch from 0x8615200 to 0x8617000, (0x1e00 bytes) 2024-09-15T00:10:08.809 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:08.813 INFO:teuthology.orchestra.run.smithi092.stdout:4549 write 0x8df5800 thru 0x8dfb32f (0x5b30 bytes) 2024-09-15T00:10:08.813 INFO:teuthology.orchestra.run.smithi092.stdout:4552 read 0x92a9600 thru 0x92ac7a7 (0x31a8 bytes) 2024-09-15T00:10:08.813 INFO:teuthology.orchestra.run.smithi092.stdout:4553 write 0xb9e5000 thru 0xb9e6232 (0x1233 bytes) 2024-09-15T00:10:08.814 INFO:teuthology.orchestra.run.smithi092.stdout:4554 read 0x5fa6200 thru 0x5fb1d98 (0xbb99 bytes) 2024-09-15T00:10:08.814 INFO:teuthology.orchestra.run.smithi092.stdout:4557 read 0x8606c00 thru 0x8607cb9 (0x10ba bytes) 2024-09-15T00:10:08.815 INFO:teuthology.orchestra.run.smithi092.stdout:4558 read 0x45e9a00 thru 0x45e9e67 (0x468 bytes) 2024-09-15T00:10:08.816 INFO:teuthology.orchestra.run.smithi092.stdout:4559 trunc from 0xd0c2400 to 0xbf60800 2024-09-15T00:10:08.840 INFO:teuthology.orchestra.run.smithi092.stdout:4561 write 0x5e9ba00 thru 0x5ea2f61 (0x7562 bytes) 2024-09-15T00:10:08.840 INFO:teuthology.orchestra.run.smithi092.stdout:4563 punch from 0xae4200 to 0xaf1600, (0xd400 bytes) 2024-09-15T00:10:08.852 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:08.851+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:08.861 INFO:teuthology.orchestra.run.smithi092.stdout:4564 write 0xede2a00 thru 0xedeabf2 (0x81f3 bytes) 2024-09-15T00:10:08.877 INFO:teuthology.orchestra.run.smithi092.stdout:4567 trunc from 0xedeac00 to 0x1d2b800 2024-09-15T00:10:08.916 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:08.914+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:08.938 INFO:teuthology.orchestra.run.smithi092.stdout:4568 write 0xbf00a00 thru 0xbf07c89 (0x728a bytes) 2024-09-15T00:10:08.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:08.941 INFO:teuthology.orchestra.run.smithi092.stdout:4569 write 0x6401a00 thru 0x6410e1f (0xf420 bytes) 2024-09-15T00:10:08.941 INFO:teuthology.orchestra.run.smithi092.stdout:4572 read 0x7bf2e00 thru 0x7bf88f9 (0x5afa bytes) 2024-09-15T00:10:08.941 INFO:teuthology.orchestra.run.smithi092.stdout:4575 write 0x89a8400 thru 0x89b3b67 (0xb768 bytes) 2024-09-15T00:10:08.941 INFO:teuthology.orchestra.run.smithi092.stdout:4578 write 0xaede600 thru 0xaee2a64 (0x4465 bytes) 2024-09-15T00:10:08.941 INFO:teuthology.orchestra.run.smithi092.stdout:4579 read 0x2db6400 thru 0x2dba7e7 (0x43e8 bytes) 2024-09-15T00:10:08.941 INFO:teuthology.orchestra.run.smithi092.stdout:4580 write 0x4779600 thru 0x4779e40 (0x841 bytes) 2024-09-15T00:10:08.971 INFO:teuthology.orchestra.run.smithi092.stdout:4581 write 0xe34f000 thru 0xe35484f (0x5850 bytes) 2024-09-15T00:10:08.972 INFO:teuthology.orchestra.run.smithi092.stdout:4583 read 0x40f4000 thru 0x40fedf1 (0xadf2 bytes) 2024-09-15T00:10:08.972 INFO:teuthology.orchestra.run.smithi092.stdout:4584 write 0x6358600 thru 0x635b11a (0x2b1b bytes) 2024-09-15T00:10:08.972 INFO:teuthology.orchestra.run.smithi092.stdout:4586 write 0x654c600 thru 0x655aa43 (0xe444 bytes) 2024-09-15T00:10:08.977 INFO:teuthology.orchestra.run.smithi092.stdout:4587 trunc from 0xe354a00 to 0xaa7b400 2024-09-15T00:10:09.006 INFO:teuthology.orchestra.run.smithi092.stdout:4588 punch from 0x7982400 to 0x798c200, (0x9e00 bytes) 2024-09-15T00:10:09.012 INFO:teuthology.orchestra.run.smithi092.stdout:4590 trunc from 0xaa7b400 to 0x6153400 2024-09-15T00:10:09.017 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:09.015+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:09.034 INFO:teuthology.orchestra.run.smithi092.stdout:skipping zero length punch hole 2024-09-15T00:10:09.034 INFO:teuthology.orchestra.run.smithi092.stdout:4592 read 0x773000 thru 0x77d970 (0xa971 bytes) 2024-09-15T00:10:09.036 INFO:teuthology.orchestra.run.smithi092.stdout:4593 read 0x3821600 thru 0x382726b (0x5c6c bytes) 2024-09-15T00:10:09.036 INFO:teuthology.orchestra.run.smithi092.stdout:4594 read 0x427e200 thru 0x428a4b7 (0xc2b8 bytes) 2024-09-15T00:10:09.036 INFO:teuthology.orchestra.run.smithi092.stdout:4595 read 0x2d47200 thru 0x2d5699b (0xf79c bytes) 2024-09-15T00:10:09.036 INFO:teuthology.orchestra.run.smithi092.stdout:4596 read 0x291fc00 thru 0x292cbb5 (0xcfb6 bytes) 2024-09-15T00:10:09.037 INFO:teuthology.orchestra.run.smithi092.stdout:4598 trunc from 0x6153400 to 0x4f3d800 2024-09-15T00:10:09.070 INFO:teuthology.orchestra.run.smithi092.stdout:4601 write 0x5d99400 thru 0x5da291d (0x951e bytes) 2024-09-15T00:10:09.075 INFO:teuthology.orchestra.run.smithi092.stdout:4605 read 0x9f4600 thru 0x9ff22e (0xac2f bytes) 2024-09-15T00:10:09.076 INFO:teuthology.orchestra.run.smithi092.stdout:4606 write 0x5a2e200 thru 0x5a35217 (0x7018 bytes) 2024-09-15T00:10:09.076 INFO:teuthology.orchestra.run.smithi092.stdout:4607 read 0x1943200 thru 0x194a8a9 (0x76aa bytes) 2024-09-15T00:10:09.077 INFO:teuthology.orchestra.run.smithi092.stdout:4608 read 0x194ae00 thru 0x195ad8b (0xff8c bytes) 2024-09-15T00:10:09.077 INFO:teuthology.orchestra.run.smithi092.stdout:4610 write 0x164b800 thru 0x164cb1b (0x131c bytes) 2024-09-15T00:10:09.079 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:10:09.088 INFO:teuthology.orchestra.run.smithi092.stdout:4613 write 0x3696c00 thru 0x36a5a67 (0xee68 bytes) 2024-09-15T00:10:09.088 INFO:teuthology.orchestra.run.smithi092.stdout:4614 read 0x2a13200 thru 0x2a1b194 (0x7f95 bytes) 2024-09-15T00:10:09.097 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:09.102 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:09.103 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:09.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:09.124 INFO:teuthology.orchestra.run.smithi092.stdout:4615 write 0xb053200 thru 0xb056e6c (0x3c6d bytes) 2024-09-15T00:10:09.129 INFO:teuthology.orchestra.run.smithi092.stdout:4617 trunc from 0xb057000 to 0x6e3ae00 2024-09-15T00:10:09.156 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:09.155+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:09.164 INFO:teuthology.orchestra.run.smithi092.stdout:4618 write 0x5bc6200 thru 0x5bc97eb (0x35ec bytes) 2024-09-15T00:10:09.165 INFO:teuthology.orchestra.run.smithi092.stdout:4620 read 0x5a9a600 thru 0x5aa3f4f (0x9950 bytes) 2024-09-15T00:10:09.165 INFO:teuthology.orchestra.run.smithi092.stdout:4621 read 0x659b200 thru 0x65a053e (0x533f bytes) 2024-09-15T00:10:09.188 INFO:teuthology.orchestra.run.smithi092.stdout:4628 write 0xe907a00 thru 0xe917662 (0xfc63 bytes) 2024-09-15T00:10:09.194 INFO:teuthology.orchestra.run.smithi092.stdout:4632 punch from 0x49a2800 to 0x49b2000, (0xf800 bytes) 2024-09-15T00:10:09.195 INFO:teuthology.orchestra.run.smithi092.stdout:4634 write 0x9ad5a00 thru 0x9ad8dc3 (0x33c4 bytes) 2024-09-15T00:10:09.196 INFO:teuthology.orchestra.run.smithi092.stdout:4639 trunc from 0xe917800 to 0xdc3b200 2024-09-15T00:10:09.220 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:09.218+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:09.224 INFO:teuthology.orchestra.run.smithi092.stdout:4640 write 0x35d7400 thru 0x35dfb57 (0x8758 bytes) 2024-09-15T00:10:09.226 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:09.226 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:09.226 INFO:teuthology.orchestra.run.smithi092.stdout:4641 write 0xafc5600 thru 0xafcd435 (0x7e36 bytes) 2024-09-15T00:10:09.226 INFO:teuthology.orchestra.run.smithi092.stdout:4644 read 0x5230200 thru 0x523f150 (0xef51 bytes) 2024-09-15T00:10:09.227 INFO:teuthology.orchestra.run.smithi092.stdout:4645 write 0xa9c3800 thru 0xa9cc476 (0x8c77 bytes) 2024-09-15T00:10:09.235 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:09.238 INFO:teuthology.orchestra.run.smithi092.stdout:4652 trunc from 0xdc3b200 to 0x3f71600 2024-09-15T00:10:09.251 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:09.270 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:09.270 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:09.277 INFO:teuthology.orchestra.run.smithi092.stdout:4653 read 0x30d3c00 thru 0x30d8191 (0x4592 bytes) 2024-09-15T00:10:09.281 INFO:teuthology.orchestra.run.smithi092.stdout:4654 trunc from 0x3f71600 to 0x753ca00 2024-09-15T00:10:09.288 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:09.289 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:09.298 INFO:teuthology.orchestra.run.smithi092.stdout:4656 trunc from 0x753ca00 to 0x4c7b000 2024-09-15T00:10:09.306 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:09.314 INFO:teuthology.orchestra.run.smithi092.stdout:4657 read 0x35c6c00 thru 0x35cfcd5 (0x90d6 bytes) 2024-09-15T00:10:09.319 INFO:teuthology.orchestra.run.smithi092.stdout:4658 read 0x4a4e00 thru 0x4b43a7 (0xf5a8 bytes) 2024-09-15T00:10:09.320 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:09.318+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:09.323 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:09.323 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:09.340 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:09.340 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:09.342 INFO:teuthology.orchestra.run.smithi092.stdout:4659 write 0xd4d8a00 thru 0xd4e3bdf (0xb1e0 bytes) 2024-09-15T00:10:09.343 INFO:teuthology.orchestra.run.smithi092.stdout:4660 read 0x1a54800 thru 0x1a581e5 (0x39e6 bytes) 2024-09-15T00:10:09.343 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:09.343 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:09.346 INFO:teuthology.orchestra.run.smithi092.stdout:4662 read 0x478b800 thru 0x4795f05 (0xa706 bytes) 2024-09-15T00:10:09.346 INFO:teuthology.orchestra.run.smithi092.stdout:4663 punch from 0x76a9600 to 0x76ae400, (0x4e00 bytes) 2024-09-15T00:10:09.346 INFO:teuthology.orchestra.run.smithi092.stdout:4665 read 0xb2a2a00 thru 0xb2acb46 (0xa147 bytes) 2024-09-15T00:10:09.347 INFO:teuthology.orchestra.run.smithi092.stdout:4666 write 0x2954400 thru 0x2958ba9 (0x47aa bytes) 2024-09-15T00:10:09.347 INFO:teuthology.orchestra.run.smithi092.stdout:4667 read 0x9644e00 thru 0x964595b (0xb5c bytes) 2024-09-15T00:10:09.347 INFO:teuthology.orchestra.run.smithi092.stdout:4668 write 0x98c3000 thru 0x98d2877 (0xf878 bytes) 2024-09-15T00:10:09.347 INFO:teuthology.orchestra.run.smithi092.stdout:4670 write 0x4199e00 thru 0x41a06c0 (0x68c1 bytes) 2024-09-15T00:10:09.347 INFO:teuthology.orchestra.run.smithi092.stdout:4671 write 0x7e02200 thru 0x7e0d3cf (0xb1d0 bytes) 2024-09-15T00:10:09.347 INFO:teuthology.orchestra.run.smithi092.stdout:4673 write 0x5406000 thru 0x540cbb2 (0x6bb3 bytes) 2024-09-15T00:10:09.348 INFO:teuthology.orchestra.run.smithi092.stdout:4674 read 0x8c97e00 thru 0x8ca22f0 (0xa4f1 bytes) 2024-09-15T00:10:09.348 INFO:teuthology.orchestra.run.smithi092.stdout:4675 read 0xabaea00 thru 0xabb5f3a (0x753b bytes) 2024-09-15T00:10:09.348 INFO:teuthology.orchestra.run.smithi092.stdout:4678 read 0x967f800 thru 0x968da3e (0xe23f bytes) 2024-09-15T00:10:09.358 INFO:teuthology.orchestra.run.smithi092.stdout:4679 trunc from 0xd4e3c00 to 0x21bd600 2024-09-15T00:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.514402793999999, 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.514364164, 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.514338227000003, 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.514322159000002, 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.514181063999999, 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.513409308999996, 2024-09-15T00:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.513324277999999, 2024-09-15T00:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.513289262000001, 2024-09-15T00:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.513263659000003, 2024-09-15T00:10:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.513089393000001, 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.512858254000001, 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.512046310000002, 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.511966569999998, 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.511761544999999, 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.511550012000001, 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.508994004999998, 2024-09-15T00:10:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.507010899000001, 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.506991327999998, 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11360 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2362650820000001, 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045212219999999997, 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11360 2024-09-15T00:10:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.46e-07 2024-09-15T00:10:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160944+0000", 2024-09-15T00:10:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0420000000000001e-06 2024-09-15T00:10:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160949+0000", 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2320000000000001e-06 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160953+0000", 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0470000000000004e-06 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.205971+0000", 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045017313000000003 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206046+0000", 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5457000000000006e-05 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206154+0000", 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010778299999999999 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2362484780000003, 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:09.516 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:09.518 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:09.516+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:09.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:09.519 INFO:teuthology.orchestra.run.smithi092.stdout:4680 trunc from 0x21bd600 to 0x5bb5800 2024-09-15T00:10:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:4681 trunc from 0x5bb5800 to 0x7816400 2024-09-15T00:10:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:4683 read 0x6d7da00 thru 0x6d86829 (0x8e2a bytes) 2024-09-15T00:10:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:4686 write 0xd445000 thru 0xd44cb4e (0x7b4f bytes) 2024-09-15T00:10:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:4687 write 0x33eaa00 thru 0x33fa8a7 (0xfea8 bytes) 2024-09-15T00:10:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:4691 read 0x6c51600 thru 0x6c5a7a8 (0x91a9 bytes) 2024-09-15T00:10:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:4693 punch from 0x3342200 to 0x334ba00, (0x9800 bytes) 2024-09-15T00:10:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:4695 write 0x53fa600 thru 0x5406fa5 (0xc9a6 bytes) 2024-09-15T00:10:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:4697 write 0xe5b8400 thru 0xe5bfce4 (0x78e5 bytes) 2024-09-15T00:10:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:4698 write 0x9830c00 thru 0x9832875 (0x1c76 bytes) 2024-09-15T00:10:09.521 INFO:teuthology.orchestra.run.smithi092.stdout:4699 punch from 0x673d400 to 0x674c400, (0xf000 bytes) 2024-09-15T00:10:09.521 INFO:teuthology.orchestra.run.smithi092.stdout:4700 punch from 0x9485200 to 0x9486800, (0x1600 bytes) 2024-09-15T00:10:09.523 INFO:teuthology.orchestra.run.smithi092.stdout:4703 trunc from 0xe5bfe00 to 0x69cf800 2024-09-15T00:10:09.556 INFO:teuthology.orchestra.run.smithi092.stdout:4708 write 0x90aa800 thru 0x90b0216 (0x5a17 bytes) 2024-09-15T00:10:09.557 INFO:teuthology.orchestra.run.smithi092.stdout:4709 punch from 0x1502200 to 0x150a400, (0x8200 bytes) 2024-09-15T00:10:09.558 INFO:teuthology.orchestra.run.smithi092.stdout:4710 punch from 0x6509000 to 0x650f400, (0x6400 bytes) 2024-09-15T00:10:09.558 INFO:teuthology.orchestra.run.smithi092.stdout:4711 write 0x7c02c00 thru 0x7c0f4f4 (0xc8f5 bytes) 2024-09-15T00:10:09.558 INFO:teuthology.orchestra.run.smithi092.stdout:4712 write 0xe9d000 thru 0xe9dec4 (0xec5 bytes) 2024-09-15T00:10:09.576 INFO:teuthology.orchestra.run.smithi092.stdout:4713 write 0xb2dc400 thru 0xb2dcbb9 (0x7ba bytes) 2024-09-15T00:10:09.580 INFO:teuthology.orchestra.run.smithi092.stdout:4714 trunc from 0xb2dcc00 to 0x855c600 2024-09-15T00:10:09.597 INFO:teuthology.orchestra.run.smithi092.stdout:4716 write 0x36f1c00 thru 0x36ffa8a (0xde8b bytes) 2024-09-15T00:10:09.618 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:09.616+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:09.618 INFO:teuthology.orchestra.run.smithi092.stdout:4719 write 0xbec2c00 thru 0xbec4595 (0x1996 bytes) 2024-09-15T00:10:09.619 INFO:teuthology.orchestra.run.smithi092.stdout:4720 write 0x3b42600 thru 0x3b42738 (0x139 bytes) 2024-09-15T00:10:09.619 INFO:teuthology.orchestra.run.smithi092.stdout:4721 flatten 2024-09-15T00:10:09.669 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:09.669 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:09.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.669 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5818 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.669 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654014+0000", 2024-09-15T00:10:09.669 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.010084539999999999, 2024-09-15T00:10:09.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010099619000000001, 2024-09-15T00:10:09.669 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5818 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654014+0000", 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654014+0000", 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654014+0000", 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0200000000000003e-07 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654016+0000", 2024-09-15T00:10:09.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9259999999999999e-06 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654022+0000", 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7529999999999998e-06 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654026+0000", 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6500000000000004e-06 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6236 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.656073+0000", 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0080247289999999995, 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0080800379999999995, 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6236 2024-09-15T00:10:09.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656073+0000", 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656073+0000", 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656074+0000", 2024-09-15T00:10:09.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2800000000000003e-07 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656075+0000", 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.683e-06 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656081+0000", 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.84e-06 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656086+0000", 2024-09-15T00:10:09.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6999999999999999e-06 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5820 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654032+0000", 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.010066308, 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010175066, 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.676 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5820 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654032+0000", 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654032+0000", 2024-09-15T00:10:09.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654032+0000", 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654034+0000", 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.77e-06 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654040+0000", 2024-09-15T00:10:09.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6740000000000002e-06 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654045+0000", 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0869999999999997e-06 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6063 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655425+0000", 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0086733839999999993, 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0088425810000000004, 2024-09-15T00:10:09.679 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6063 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655425+0000", 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655425+0000", 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655425+0000", 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8599999999999999e-07 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655427+0000", 2024-09-15T00:10:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.852e-06 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655433+0000", 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6629999999999998e-06 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655438+0000", 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9869999999999999e-06 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5821 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654051+0000", 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.010046862, 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010273018, 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5821 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.683 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654051+0000", 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654051+0000", 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654052+0000", 2024-09-15T00:10:09.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654058+0000", 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1199999999999999e-06 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654066+0000", 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8180000000000004e-06 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654071+0000", 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6879999999999996e-06 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6065 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655444+0000", 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0086545449999999996, 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0089392870000000006, 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.686 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6065 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655444+0000", 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655444+0000", 2024-09-15T00:10:09.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655444+0000", 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1799999999999995e-07 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655446+0000", 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9149999999999999e-06 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655452+0000", 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8989999999999998e-06 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655459+0000", 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.849e-06 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5822 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.689 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654100+0000", 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0099977820000000002, 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010340004999999999, 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5822 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654100+0000", 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654100+0000", 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654101+0000", 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.82e-07 2024-09-15T00:10:09.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654103+0000", 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4389999999999999e-06 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654111+0000", 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2949999999999996e-06 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654116+0000", 2024-09-15T00:10:09.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.502e-06 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6066 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655465+0000", 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0086327929999999997, 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0090062159999999992, 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6066 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655465+0000", 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655465+0000", 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655466+0000", 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655468+0000", 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.013e-06 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655474+0000", 2024-09-15T00:10:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0079999999999999e-06 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655479+0000", 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7890000000000002e-06 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5824 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654124+0000", 2024-09-15T00:10:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0099745649999999995, 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010376106, 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5824 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654124+0000", 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654124+0000", 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654124+0000", 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654126+0000", 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654132+0000", 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654137+0000", 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7010000000000004e-06 2024-09-15T00:10:09.699 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6067 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655484+0000", 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0086139890000000007, 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0090481799999999994, 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.700 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6067 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655484+0000", 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655484+0000", 2024-09-15T00:10:09.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655485+0000", 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655486+0000", 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9089999999999998e-06 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655492+0000", 2024-09-15T00:10:09.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.694e-06 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655498+0000", 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5339999999999999e-06 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5825 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654142+0000", 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0099560710000000004, 2024-09-15T00:10:09.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01041799, 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5825 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654142+0000", 2024-09-15T00:10:09.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654142+0000", 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654142+0000", 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654144+0000", 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8020000000000001e-06 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654150+0000", 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6550000000000001e-06 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654154+0000", 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.583e-06 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6069 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655505+0000", 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0085934600000000007, 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0090836809999999997, 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6069 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.707 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655505+0000", 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655505+0000", 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655505+0000", 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8299999999999997e-07 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655507+0000", 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1169999999999998e-06 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655514+0000", 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4559999999999997e-06 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655519+0000", 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.288e-06 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5826 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654160+0000", 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0099380079999999999, 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010460270000000001, 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5826 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654160+0000", 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654160+0000", 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654161+0000", 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2399999999999999e-07 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654163+0000", 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9829999999999999e-06 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654169+0000", 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3180000000000003e-06 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654175+0000", 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9900000000000002e-06 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6070 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655540+0000", 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0085578689999999992, 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0091307409999999995, 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6070 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655540+0000", 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655540+0000", 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655541+0000", 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3799999999999997e-07 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655543+0000", 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.464e-06 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655550+0000", 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8179999999999998e-06 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655555+0000", 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3319999999999999e-06 2024-09-15T00:10:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5827 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654181+0000", 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0099169319999999998, 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010547348999999999, 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5827 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654181+0000", 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654181+0000", 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654182+0000", 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654184+0000", 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0049999999999999e-06 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654190+0000", 2024-09-15T00:10:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9429999999999997e-06 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654194+0000", 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6979999999999997e-06 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6071 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655562+0000", 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0085361010000000008, 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0092241470000000002, 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6071 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655562+0000", 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655562+0000", 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655562+0000", 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655564+0000", 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9769999999999999e-06 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655570+0000", 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655575+0000", 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5829 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654200+0000", 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0098983649999999992, 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010642548, 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5829 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654200+0000", 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654200+0000", 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654200+0000", 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654202+0000", 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8369999999999999e-06 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654208+0000", 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7429999999999999e-06 2024-09-15T00:10:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654212+0000", 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5210000000000004e-06 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6072 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655580+0000", 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0085178089999999995, 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0093108789999999993, 2024-09-15T00:10:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6072 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655580+0000", 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655580+0000", 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655581+0000", 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655583+0000", 2024-09-15T00:10:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9470000000000002e-06 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655588+0000", 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.874e-06 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655594+0000", 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.04e-06 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6014 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654916+0000", 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0091825629999999995, 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010004602, 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6014 2024-09-15T00:10:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654916+0000", 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654916+0000", 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654916+0000", 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654918+0000", 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2419999999999999e-06 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654925+0000", 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7939999999999997e-06 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654931+0000", 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6019999999999999e-06 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6015 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654937+0000", 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.009161074, 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010013396000000001, 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6015 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654937+0000", 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654937+0000", 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654937+0000", 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0999999999999999e-07 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654940+0000", 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.261e-06 2024-09-15T00:10:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654946+0000", 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654950+0000", 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7589999999999997e-06 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6016 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654956+0000", 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0091422050000000005, 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010022603, 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6016 2024-09-15T00:10:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654956+0000", 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654956+0000", 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654956+0000", 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654958+0000", 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9180000000000001e-06 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654964+0000", 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6049999999999997e-06 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654968+0000", 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5299999999999998e-06 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6017 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654975+0000", 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0091231109999999997, 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01003108, 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6017 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654975+0000", 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654975+0000", 2024-09-15T00:10:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654976+0000", 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1600000000000001e-07 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654978+0000", 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.119e-06 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654986+0000", 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3420000000000008e-06 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654991+0000", 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2480000000000004e-06 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6019 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654998+0000", 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0091004069999999996, 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010036447, 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6019 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654998+0000", 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654998+0000", 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654998+0000", 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9399999999999995e-07 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655000+0000", 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0310000000000001e-06 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655006+0000", 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9939999999999997e-06 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655011+0000", 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7319999999999998e-06 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6021 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655018+0000", 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0090803040000000008, 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010045346, 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6021 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655018+0000", 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655018+0000", 2024-09-15T00:10:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655018+0000", 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0200000000000003e-07 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655020+0000", 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9549999999999999e-06 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655026+0000", 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9240000000000004e-06 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655032+0000", 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6189999999999999e-06 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6023 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655058+0000", 2024-09-15T00:10:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0090399580000000007, 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010058803, 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6023 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655058+0000", 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655058+0000", 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655059+0000", 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1200000000000003e-07 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655061+0000", 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1299999999999999e-06 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655067+0000", 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6229999999999999e-06 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655073+0000", 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.118e-06 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6025 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655079+0000", 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0090193080000000002, 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010095376, 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6025 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655079+0000", 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655079+0000", 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655079+0000", 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9299999999999998e-07 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655081+0000", 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9209999999999999e-06 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655087+0000", 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7729999999999996e-06 2024-09-15T00:10:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655092+0000", 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6639999999999997e-06 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6027 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655097+0000", 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0090008680000000004, 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010133631000000001, 2024-09-15T00:10:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6027 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655097+0000", 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655097+0000", 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655098+0000", 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655100+0000", 2024-09-15T00:10:09.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8929999999999999e-06 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655105+0000", 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8969999999999996e-06 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655111+0000", 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8989999999999998e-06 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6028 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655118+0000", 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0089801559999999996, 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010169460999999999, 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6028 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.762 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655118+0000", 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655118+0000", 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655118+0000", 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1600000000000002e-07 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655125+0000", 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3389999999999998e-06 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655131+0000", 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7309999999999996e-06 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655137+0000", 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1540000000000001e-06 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6030 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655144+0000", 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0089542379999999998, 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010201019, 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6030 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655144+0000", 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655144+0000", 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655144+0000", 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5700000000000002e-07 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655147+0000", 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2409999999999998e-06 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655153+0000", 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2330000000000003e-06 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655158+0000", 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0499999999999999e-06 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6032 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655164+0000", 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0089345789999999998, 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010215020999999999, 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6032 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.769 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655164+0000", 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655164+0000", 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655164+0000", 2024-09-15T00:10:09.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7099999999999997e-07 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655166+0000", 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9420000000000002e-06 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655172+0000", 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0379999999999996e-06 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655177+0000", 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5860 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654489+0000", 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0096095390000000003, 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010926237, 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.772 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5860 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654489+0000", 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654489+0000", 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654489+0000", 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3700000000000001e-07 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654491+0000", 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0379999999999998e-06 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654498+0000", 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4109999999999999e-06 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654504+0000", 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2700000000000004e-06 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6217 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655851+0000", 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.008247318, 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0095926610000000006, 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6217 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655851+0000", 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655851+0000", 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655851+0000", 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.15e-07 2024-09-15T00:10:09.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655853+0000", 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8610000000000001e-06 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655859+0000", 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6790000000000002e-06 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655864+0000", 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.446e-06 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5861 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654511+0000", 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0095874359999999995, 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010963443, 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5861 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654511+0000", 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654511+0000", 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654511+0000", 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9099999999999999e-07 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654513+0000", 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1799999999999999e-06 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654519+0000", 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.981e-06 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654524+0000", 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9439999999999996e-06 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6218 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.782 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655870+0000", 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0082283300000000007, 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0096326620000000002, 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6218 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655870+0000", 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655870+0000", 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655870+0000", 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9000000000000002e-07 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655877+0000", 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8310000000000003e-06 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655883+0000", 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0249999999999999e-06 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655888+0000", 2024-09-15T00:10:09.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9359999999999998e-06 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5862 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654530+0000", 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.00956831, 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011009267, 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5862 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654530+0000", 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654530+0000", 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654530+0000", 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654532+0000", 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9479999999999998e-06 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654538+0000", 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7150000000000003e-06 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654543+0000", 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7829999999999998e-06 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6220 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655894+0000", 2024-09-15T00:10:09.789 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0082040050000000003, 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0096942339999999995, 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6220 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655894+0000", 2024-09-15T00:10:09.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655894+0000", 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655894+0000", 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655896+0000", 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8670000000000001e-06 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655902+0000", 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.789e-06 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655907+0000", 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7249999999999997e-06 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.792 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5863 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654548+0000", 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0095500470000000007, 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011096844999999999, 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5863 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654548+0000", 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654548+0000", 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654548+0000", 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6300000000000001e-07 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654550+0000", 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.849e-06 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654556+0000", 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.981e-06 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654563+0000", 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6780000000000003e-06 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6222 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655913+0000", 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0081855109999999995, 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0097888479999999993, 2024-09-15T00:10:09.796 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6222 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655913+0000", 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655913+0000", 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655913+0000", 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7e-07 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655915+0000", 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.863e-06 2024-09-15T00:10:09.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655921+0000", 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9399999999999999e-06 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655926+0000", 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.87e-06 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5865 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654581+0000", 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0095171360000000007, 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011179969, 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5865 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654581+0000", 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654581+0000", 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654582+0000", 2024-09-15T00:10:09.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.44e-07 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654584+0000", 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1150000000000001e-06 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654592+0000", 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1149999999999994e-06 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654598+0000", 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7540000000000003e-06 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6224 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655931+0000", 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0081669580000000002, 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0098843090000000008, 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.803 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6224 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655931+0000", 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655931+0000", 2024-09-15T00:10:09.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655931+0000", 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655933+0000", 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9e-06 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655939+0000", 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7849999999999997e-06 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655944+0000", 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6840000000000004e-06 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5866 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654607+0000", 2024-09-15T00:10:09.806 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0094915150000000007, 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011238522000000001, 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5866 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654607+0000", 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654607+0000", 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654607+0000", 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4299999999999998e-07 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654615+0000", 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7880000000000007e-06 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654621+0000", 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.545e-06 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654627+0000", 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7699999999999998e-06 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6226 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655949+0000", 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0081486560000000006, 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0099234969999999999, 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6226 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.810 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655949+0000", 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655949+0000", 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655950+0000", 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0699999999999998e-07 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655952+0000", 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8840000000000001e-06 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655958+0000", 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.947e-06 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655963+0000", 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2490000000000001e-06 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5867 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654633+0000", 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0094647340000000007, 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011271203, 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5867 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654633+0000", 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654633+0000", 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654634+0000", 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3600000000000004e-07 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654636+0000", 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8300000000000001e-06 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654642+0000", 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7610000000000004e-06 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654646+0000", 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7230000000000004e-06 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.816 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6227 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655969+0000", 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0081288999999999997, 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0099899859999999993, 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6227 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655969+0000", 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655969+0000", 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655970+0000", 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4999999999999998e-07 2024-09-15T00:10:09.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655972+0000", 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9e-06 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655978+0000", 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2090000000000002e-06 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655983+0000", 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:09.819 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:5868 3.6 3.1f6cf2e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.654652+0000", 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0094463069999999993, 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011365081000000001, 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.820 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 5868 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654652+0000", 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654652+0000", 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654652+0000", 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654654+0000", 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8640000000000001e-06 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654660+0000", 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.852e-06 2024-09-15T00:10:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.654665+0000", 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7129999999999996e-06 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6229 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.655989+0000", 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0081091570000000005, 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010083973, 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6229 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655989+0000", 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655989+0000", 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655989+0000", 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1699999999999998e-07 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655992+0000", 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0320000000000002e-06 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.655999+0000", 2024-09-15T00:10:09.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2189999999999998e-06 2024-09-15T00:10:09.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656004+0000", 2024-09-15T00:10:09.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9169999999999997e-06 2024-09-15T00:10:09.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.827 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6230 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.656017+0000", 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0080808709999999999, 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010113403, 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.828 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6230 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656017+0000", 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656017+0000", 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656018+0000", 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656020+0000", 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.165e-06 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656026+0000", 2024-09-15T00:10:09.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4250000000000003e-06 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656032+0000", 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1699999999999996e-06 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6232 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.656038+0000", 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0080605139999999995, 2024-09-15T00:10:09.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010147985, 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6232 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656038+0000", 2024-09-15T00:10:09.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656038+0000", 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656038+0000", 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4200000000000001e-07 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656040+0000", 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.75e-06 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656046+0000", 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7230000000000001e-06 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656050+0000", 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6110000000000004e-06 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:6234 3.6 3.2697b5e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:09.656056+0000", 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0080425789999999994, 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.010186653, 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 6234 2024-09-15T00:10:09.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656056+0000", 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656056+0000", 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:09.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656056+0000", 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7500000000000001e-07 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656058+0000", 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7439999999999999e-06 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656063+0000", 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8370000000000002e-06 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:09.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:09.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:09.656068+0000", 2024-09-15T00:10:09.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6140000000000002e-06 2024-09-15T00:10:09.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.838 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:09.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:09.838 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:09.838 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 49 2024-09-15T00:10:09.838 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:09.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:09.842 INFO:teuthology.orchestra.run.smithi092.stdout:4722 write 0x6f81400 thru 0x6f8bd34 (0xa935 bytes) 2024-09-15T00:10:09.842 INFO:teuthology.orchestra.run.smithi092.stdout:4725 punch from 0xc26e00 to 0xc28200, (0x1400 bytes) 2024-09-15T00:10:09.842 INFO:teuthology.orchestra.run.smithi092.stdout:4727 read 0x66bde00 thru 0x66c9184 (0xb385 bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4729 write 0x7158a00 thru 0x7167f63 (0xf564 bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4730 write 0x8aa2a00 thru 0x8aae701 (0xbd02 bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4732 punch from 0x9479000 to 0x9481a00, (0x8a00 bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4734 punch from 0x93c5200 to 0x93cf200, (0xa000 bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4736 read 0x594bc00 thru 0x595bae4 (0xfee5 bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4737 write 0x57e3e00 thru 0x57eacaa (0x6eab bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4740 read 0x398aa00 thru 0x398b83a (0xe3b bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4741 read 0xad5e600 thru 0xad5fabd (0x14be bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4742 write 0x6c43600 thru 0x6c45be9 (0x25ea bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4743 write 0x5963200 thru 0x5970dcc (0xdbcd bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4744 trunc from 0xbec4600 to 0x1ea9e00 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4745 read 0x299c00 thru 0x2a5df1 (0xc1f2 bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4747 read 0x164cc00 thru 0x164eaad (0x1eae bytes) 2024-09-15T00:10:09.843 INFO:teuthology.orchestra.run.smithi092.stdout:4750 read 0x1288000 thru 0x1294863 (0xc864 bytes) 2024-09-15T00:10:09.844 INFO:teuthology.orchestra.run.smithi092.stdout:4751 trunc from 0x1ea9e00 to 0xc8d2200 2024-09-15T00:10:09.844 INFO:teuthology.orchestra.run.smithi092.stdout:4756 read 0x4632000 thru 0x4636dac (0x4dad bytes) 2024-09-15T00:10:09.844 INFO:teuthology.orchestra.run.smithi092.stdout:4757 read 0x72bdc00 thru 0x72c9ef8 (0xc2f9 bytes) 2024-09-15T00:10:09.844 INFO:teuthology.orchestra.run.smithi092.stdout:4758 read 0xb4a7400 thru 0xb4b31a5 (0xbda6 bytes) 2024-09-15T00:10:09.844 INFO:teuthology.orchestra.run.smithi092.stdout:4759 write 0x867000 thru 0x869cbb (0x2cbc bytes) 2024-09-15T00:10:09.844 INFO:teuthology.orchestra.run.smithi092.stdout:4760 read 0x3fda600 thru 0x3fe47f8 (0xa1f9 bytes) 2024-09-15T00:10:09.844 INFO:teuthology.orchestra.run.smithi092.stdout:4761 read 0x1993600 thru 0x1999997 (0x6398 bytes) 2024-09-15T00:10:09.844 INFO:teuthology.orchestra.run.smithi092.stdout:4763 write 0xd8a3e00 thru 0xd8b1d5c (0xdf5d bytes) 2024-09-15T00:10:09.845 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:09.838+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:09.847 INFO:teuthology.orchestra.run.smithi092.stdout:4764 trunc from 0xd8b1e00 to 0xbd5a00 2024-09-15T00:10:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:4766 read 0x2fc000 thru 0x2fea52 (0x2a53 bytes) 2024-09-15T00:10:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:4767 punch from 0x75dc00 to 0x75f600, (0x1a00 bytes) 2024-09-15T00:10:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:4769 trunc from 0xbd5a00 to 0x1b6f000 2024-09-15T00:10:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:4772 read 0x1a72a00 thru 0x1a74873 (0x1e74 bytes) 2024-09-15T00:10:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:4775 read 0x56b400 thru 0x577a09 (0xc60a bytes) 2024-09-15T00:10:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:4776 trunc from 0x1b6f000 to 0x5cc1800 2024-09-15T00:10:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:4777 trunc from 0x5cc1800 to 0x5f3b800 2024-09-15T00:10:09.933 INFO:teuthology.orchestra.run.smithi092.stdout:4781 read 0xc0d000 thru 0xc199d6 (0xc9d7 bytes) 2024-09-15T00:10:09.933 INFO:teuthology.orchestra.run.smithi092.stdout:4782 read 0x109e200 thru 0x109f880 (0x1681 bytes) 2024-09-15T00:10:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:4783 trunc from 0x5f3b800 to 0x86c1600 2024-09-15T00:10:09.940 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:09.939+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:4784 punch from 0x70d6c00 to 0x70e5e00, (0xf200 bytes) 2024-09-15T00:10:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:4785 write 0x62b7000 thru 0x62c4de4 (0xdde5 bytes) 2024-09-15T00:10:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:4786 punch from 0x7eb6600 to 0x7ec4800, (0xe200 bytes) 2024-09-15T00:10:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:4788 punch from 0x6bb8e00 to 0x6bc8400, (0xf600 bytes) 2024-09-15T00:10:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:4789 trunc from 0x86c1600 to 0x4bd7c00 2024-09-15T00:10:09.961 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:09.961 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:09.961 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:09.961 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:09.961 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:09.971 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:4790 punch from 0x210600 to 0x213c00, (0x3600 bytes) 2024-09-15T00:10:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:4792 write 0xe66f800 thru 0xe6762f0 (0x6af1 bytes) 2024-09-15T00:10:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:4793 write 0x6c39600 thru 0x6c3f970 (0x6371 bytes) 2024-09-15T00:10:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:4796 punch from 0x95f4a00 to 0x95ffe00, (0xb400 bytes) 2024-09-15T00:10:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:4797 trunc from 0xe676400 to 0x28be200 2024-09-15T00:10:10.041 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:10.039+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:4798 punch from 0xf96a00 to 0xfa1e00, (0xb400 bytes) 2024-09-15T00:10:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:4803 punch from 0x1f23c00 to 0x1f2f800, (0xbc00 bytes) 2024-09-15T00:10:10.057 INFO:teuthology.orchestra.run.smithi092.stdout:4806 trunc from 0x28be200 to 0x7313400 2024-09-15T00:10:10.073 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-15T00:10:10.073 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:10:10.073 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-15T00:10:10.073 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T00:10:10.073 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T00:10:10.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-15T00:10:10.078 INFO:teuthology.orchestra.run.smithi092.stdout:4807 trunc from 0x7313400 to 0x1523e00 2024-09-15T00:10:10.089 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059997342999999, 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.090 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.091 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.091 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:10:10.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.091 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:10.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059977087999997, 2024-09-15T00:10:10.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:10.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:10.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059957341999997, 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.100 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.104 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059937392000002, 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:10.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059909892, 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:10.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059893051000003, 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:10.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:10.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059877053000001, 2024-09-15T00:10:10.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:10.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059854065000003, 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:10.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059828181, 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:10.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:10.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:10.132 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059811754000002, 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:10.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:10.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059797482999997, 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:10.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.141 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.059788165999997, 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.142 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:10.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.146 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.058119154000003, 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:10.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.058084090999998, 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.151 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:10.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.058059063999998, 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.058034755999998, 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2674 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3768320~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.055888410000001, 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059850591000000002, 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2674 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.165 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031671+0000", 2024-09-15T00:10:10.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031682+0000", 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1248e-05 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031693+0000", 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1395e-05 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031705+0000", 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2106000000000001e-05 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091352+0000", 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059646421999999998 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091441+0000", 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9174999999999999e-05 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091521+0000", 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9636999999999997e-05 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.055842505000001, 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:10:10.169 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:10:10.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.173 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.055818373000001, 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:10.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.178 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.055789001000001, 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:10.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.183 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.183 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:10.184 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:10.182+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:10.186 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:10.187 INFO:teuthology.orchestra.run.smithi092.stdout:4808 punch from 0x28c00 to 0x2f000, (0x6400 bytes) 2024-09-15T00:10:10.187 INFO:teuthology.orchestra.run.smithi092.stdout:4811 read 0x870e00 thru 0x8714f9 (0x6fa bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4812 read 0xad5a00 thru 0xadde1e (0x841f bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4814 punch from 0x6fbe00 to 0x700800, (0x4a00 bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4815 write 0x454b600 thru 0x455296d (0x736e bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4817 write 0xcd9f800 thru 0xcda3668 (0x3e69 bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4822 write 0x85a7000 thru 0x85af176 (0x8177 bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4826 punch from 0x6614000 to 0x661b200, (0x7200 bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4828 punch from 0x35aac00 to 0x35ae200, (0x3600 bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4829 read 0x5aefa00 thru 0x5afc087 (0xc688 bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4830 read 0x37a3000 thru 0x37ad111 (0xa112 bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4832 write 0x716ea00 thru 0x71747e7 (0x5de8 bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4834 punch from 0x93e2200 to 0x93ef600, (0xd400 bytes) 2024-09-15T00:10:10.188 INFO:teuthology.orchestra.run.smithi092.stdout:4836 punch from 0x3f24000 to 0x3f2c200, (0x8200 bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4837 write 0x7ea4e00 thru 0x7eb00d5 (0xb2d6 bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4838 write 0x619c600 thru 0x61abeeb (0xf8ec bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4839 read 0x8e9ae00 thru 0x8e9bf19 (0x111a bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4840 punch from 0x1a7ca00 to 0x1a81a00, (0x5000 bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4842 write 0x3faa00 thru 0x401a5c (0x705d bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4846 write 0x6311800 thru 0x631d6da (0xbedb bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4847 read 0x9be8c00 thru 0x9bf23c2 (0x97c3 bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4848 read 0x67bb200 thru 0x67c9073 (0xde74 bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4849 write 0x83a5e00 thru 0x83b206d (0xc26e bytes) 2024-09-15T00:10:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:4850 trunc from 0xcda3800 to 0xc786e00 2024-09-15T00:10:10.197 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:10.197 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T00:10:10.197 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:10.210 INFO:teuthology.orchestra.run.smithi092.stdout:4853 punch from 0x9307600 to 0x9316600, (0xf000 bytes) 2024-09-15T00:10:10.212 INFO:teuthology.orchestra.run.smithi092.stdout:4854 write 0xb29fc00 thru 0xb2a43a9 (0x47aa bytes) 2024-09-15T00:10:10.236 INFO:teuthology.orchestra.run.smithi092.stdout:4855 write 0xdbe2800 thru 0xdbe52eb (0x2aec bytes) 2024-09-15T00:10:10.236 INFO:teuthology.orchestra.run.smithi092.stdout:4856 read 0xc139e00 thru 0xc144be1 (0xade2 bytes) 2024-09-15T00:10:10.236 INFO:teuthology.orchestra.run.smithi092.stdout:4857 write 0x234fa00 thru 0x2355419 (0x5a1a bytes) 2024-09-15T00:10:10.236 INFO:teuthology.orchestra.run.smithi092.stdout:4860 read 0x7e52800 thru 0x7e5a478 (0x7c79 bytes) 2024-09-15T00:10:10.236 INFO:teuthology.orchestra.run.smithi092.stdout:4861 read 0x1e93e00 thru 0x1e987b7 (0x49b8 bytes) 2024-09-15T00:10:10.242 INFO:teuthology.orchestra.run.smithi092.stdout:4862 trunc from 0xdbe5400 to 0x7a2b400 2024-09-15T00:10:10.279 INFO:teuthology.orchestra.run.smithi092.stdout:4863 write 0x1f4d400 thru 0x1f5c339 (0xef3a bytes) 2024-09-15T00:10:10.285 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:10.284+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:10.286 INFO:teuthology.orchestra.run.smithi092.stdout:4865 trunc from 0x7a2b400 to 0x1b44400 2024-09-15T00:10:10.320 INFO:teuthology.orchestra.run.smithi092.stdout:4866 read 0x3d5600 thru 0x3e55ee (0xffef bytes) 2024-09-15T00:10:10.321 INFO:teuthology.orchestra.run.smithi092.stdout:4867 punch from 0x100b800 to 0x100da00, (0x2200 bytes) 2024-09-15T00:10:10.321 INFO:teuthology.orchestra.run.smithi092.stdout:4869 punch from 0x11d6e00 to 0x11dbc00, (0x4e00 bytes) 2024-09-15T00:10:10.326 INFO:teuthology.orchestra.run.smithi092.stdout:4870 trunc from 0x1b44400 to 0x5f93800 2024-09-15T00:10:10.340 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:10.340 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:10.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.340 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5195.0:8667 3.7 3:e24420bb:::rbd_data.1129b69d8a66.0000000000000000:head [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-15T00:10:10.340 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:10.339224+0000", 2024-09-15T00:10:10.340 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.4689000000000004e-05, 2024-09-15T00:10:10.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0481000000000004e-05, 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5195", 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/209744101", 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 8667 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:10.339224+0000", 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:10.339224+0000", 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:10.339225+0000", 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0200000000000002e-07 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:10.339227+0000", 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6709999999999999e-06 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:10.339234+0000", 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7680000000000003e-06 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:10.339238+0000", 2024-09-15T00:10:10.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7230000000000002e-06 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:10.339275+0000", 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6874e-05 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 1 2024-09-15T00:10:10.344 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:10.345 INFO:teuthology.orchestra.run.smithi092.stdout:4873 write 0x4789600 thru 0x4797303 (0xdd04 bytes) 2024-09-15T00:10:10.350 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:10.351 INFO:teuthology.orchestra.run.smithi092.stdout:4875 trunc from 0x5f93800 to 0xd959e00 2024-09-15T00:10:10.369 INFO:teuthology.orchestra.run.smithi092.stdout:4876 read 0x68cae00 thru 0x68d8119 (0xd31a bytes) 2024-09-15T00:10:10.369 INFO:teuthology.orchestra.run.smithi092.stdout:4877 punch from 0xcb60600 to 0xcb6f600, (0xf000 bytes) 2024-09-15T00:10:10.372 INFO:teuthology.orchestra.run.smithi092.stdout:4878 trunc from 0xd959e00 to 0xae04e00 2024-09-15T00:10:10.394 INFO:teuthology.orchestra.run.smithi092.stdout:4879 punch from 0xdc8200 to 0xdd3800, (0xb600 bytes) 2024-09-15T00:10:10.394 INFO:teuthology.orchestra.run.smithi092.stdout:4881 write 0x8fbae00 thru 0x8fbd5db (0x27dc bytes) 2024-09-15T00:10:10.394 INFO:teuthology.orchestra.run.smithi092.stdout:4884 read 0x7753400 thru 0x775ab20 (0x7721 bytes) 2024-09-15T00:10:10.394 INFO:teuthology.orchestra.run.smithi092.stdout:4886 read 0x17d4800 thru 0x17d74f2 (0x2cf3 bytes) 2024-09-15T00:10:10.394 INFO:teuthology.orchestra.run.smithi092.stdout:4887 read 0x5063400 thru 0x506a20d (0x6e0e bytes) 2024-09-15T00:10:10.395 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:10.392+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:10.416 INFO:teuthology.orchestra.run.smithi092.stdout:4888 write 0xd659800 thru 0xd65bfef (0x27f0 bytes) 2024-09-15T00:10:10.416 INFO:teuthology.orchestra.run.smithi092.stdout:4892 read 0x7050a00 thru 0x705d48d (0xca8e bytes) 2024-09-15T00:10:10.416 INFO:teuthology.orchestra.run.smithi092.stdout:4893 write 0x1f8f600 thru 0x1f9dd95 (0xe796 bytes) 2024-09-15T00:10:10.416 INFO:teuthology.orchestra.run.smithi092.stdout:4897 read 0x5966c00 thru 0x596a532 (0x3933 bytes) 2024-09-15T00:10:10.416 INFO:teuthology.orchestra.run.smithi092.stdout:4898 read 0xc53d000 thru 0xc543a0e (0x6a0f bytes) 2024-09-15T00:10:10.429 INFO:teuthology.orchestra.run.smithi092.stdout:4899 trunc from 0xd65c000 to 0x1f37600 2024-09-15T00:10:10.453 INFO:teuthology.orchestra.run.smithi092.stdout:4901 punch from 0xf39200 to 0xf3fa00, (0x6800 bytes) 2024-09-15T00:10:10.455 INFO:teuthology.orchestra.run.smithi092.stdout:4903 punch from 0x1147c00 to 0x114cc00, (0x5000 bytes) 2024-09-15T00:10:10.455 INFO:teuthology.orchestra.run.smithi092.stdout:4904 read 0x8bcc00 thru 0x8cb763 (0xeb64 bytes) 2024-09-15T00:10:10.458 INFO:teuthology.orchestra.run.smithi092.stdout:4905 read 0x142fe00 thru 0x1439c63 (0x9e64 bytes) 2024-09-15T00:10:10.465 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:10.465 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:10.466 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:10.466 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:10.466 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:10.468 INFO:teuthology.orchestra.run.smithi092.stdout:4907 write 0x3862e00 thru 0x386711b (0x431c bytes) 2024-09-15T00:10:10.470 INFO:teuthology.orchestra.run.smithi092.stdout:4911 trunc from 0x3867200 to 0x50d1c00 2024-09-15T00:10:10.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:10.489 INFO:teuthology.orchestra.run.smithi092.stdout:4912 trunc from 0x50d1c00 to 0x2cea200 2024-09-15T00:10:10.502 INFO:teuthology.orchestra.run.smithi092.stdout:4915 write 0x26bfe00 thru 0x26cabca (0xadcb bytes) 2024-09-15T00:10:10.503 INFO:teuthology.orchestra.run.smithi092.stdout:4917 read 0xee1400 thru 0xee9e85 (0x8a86 bytes) 2024-09-15T00:10:10.503 INFO:teuthology.orchestra.run.smithi092.stdout:4918 read 0xdb9400 thru 0xdb94ac (0xad bytes) 2024-09-15T00:10:10.503 INFO:teuthology.orchestra.run.smithi092.stdout:4919 read 0x29c7000 thru 0x29d5c0a (0xec0b bytes) 2024-09-15T00:10:10.516 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:10.514+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:10.534 INFO:teuthology.orchestra.run.smithi092.stdout:4922 write 0xdb0c600 thru 0xdb14d5c (0x875d bytes) 2024-09-15T00:10:10.535 INFO:teuthology.orchestra.run.smithi092.stdout:4923 punch from 0xad39800 to 0xad45000, (0xb800 bytes) 2024-09-15T00:10:10.547 INFO:teuthology.orchestra.run.smithi092.stdout:4925 trunc from 0xdb14e00 to 0x3bd6400 2024-09-15T00:10:10.574 INFO:teuthology.orchestra.run.smithi092.stdout:4926 write 0x9314800 thru 0x9316320 (0x1b21 bytes) 2024-09-15T00:10:10.587 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:10.585+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:10.590 INFO:teuthology.orchestra.run.smithi092.stdout:4930 write 0xba7a800 thru 0xba7ffd6 (0x57d7 bytes) 2024-09-15T00:10:10.592 INFO:teuthology.orchestra.run.smithi092.stdout:4931 trunc from 0xba80000 to 0xcc6a000 2024-09-15T00:10:10.606 INFO:teuthology.orchestra.run.smithi092.stdout:4932 read 0x202400 thru 0x206b9c (0x479d bytes) 2024-09-15T00:10:10.608 INFO:teuthology.orchestra.run.smithi092.stdout:4933 write 0x5a18a00 thru 0x5a197c7 (0xdc8 bytes) 2024-09-15T00:10:10.608 INFO:teuthology.orchestra.run.smithi092.stdout:4934 write 0x8756e00 thru 0x8762304 (0xb505 bytes) 2024-09-15T00:10:10.608 INFO:teuthology.orchestra.run.smithi092.stdout:4935 write 0x915000 thru 0x91e20c (0x920d bytes) 2024-09-15T00:10:10.609 INFO:teuthology.orchestra.run.smithi092.stdout:4936 trunc from 0xcc6a000 to 0xc0f0000 2024-09-15T00:10:10.629 INFO:teuthology.orchestra.run.smithi092.stdout:4937 read 0x3396e00 thru 0x339a771 (0x3972 bytes) 2024-09-15T00:10:10.632 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:10.632 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:10.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:10.632 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:10.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.632 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.632 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:10.632 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.604102361000002, 2024-09-15T00:10:10.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:10.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:10.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:10.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:10.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:10.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.604080443000001, 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:10.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:10.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:10.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:10.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:10.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:10.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:10.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:10.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.644 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.604059171000003, 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:10.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.602317911, 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.602296004999999, 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:10.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.659 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.602273912999998, 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.660 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:10.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:10.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.602242173999997, 2024-09-15T00:10:10.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:10.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.601943970999997, 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.670 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:10.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.674 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.601910265999997, 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:10.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.679 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.597608192999999, 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.680 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:10.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:10.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:10.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:10.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.684 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.597283322999999, 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:10.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.596872918000003, 2024-09-15T00:10:10.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:10.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.596818014999997, 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:10.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:10.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.59678598, 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:10.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:10.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.704 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.595308150000001, 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:10.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.595286434999998, 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:10.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.595264956000001, 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.593251852000002, 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.593232211, 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.593185663, 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:10.733 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:10.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:10.736 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:10.733+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:10.737 INFO:teuthology.orchestra.run.smithi092.stdout:4938 write 0xc2f9c00 thru 0xc308c98 (0xf099 bytes) 2024-09-15T00:10:10.737 INFO:teuthology.orchestra.run.smithi092.stdout:4939 punch from 0x9091000 to 0x909b200, (0xa200 bytes) 2024-09-15T00:10:10.737 INFO:teuthology.orchestra.run.smithi092.stdout:4940 write 0x952e00 thru 0x9578b0 (0x4ab1 bytes) 2024-09-15T00:10:10.737 INFO:teuthology.orchestra.run.smithi092.stdout:4941 punch from 0xffe200 to 0x1009800, (0xb600 bytes) 2024-09-15T00:10:10.737 INFO:teuthology.orchestra.run.smithi092.stdout:4942 trunc from 0xc308e00 to 0x4e1de00 2024-09-15T00:10:10.737 INFO:teuthology.orchestra.run.smithi092.stdout:4946 write 0xd61cc00 thru 0xd62bcc7 (0xf0c8 bytes) 2024-09-15T00:10:10.737 INFO:teuthology.orchestra.run.smithi092.stdout:4949 write 0x25b9a00 thru 0x25bd075 (0x3676 bytes) 2024-09-15T00:10:10.738 INFO:teuthology.orchestra.run.smithi092.stdout:4951 write 0xd448600 thru 0xd453b4f (0xb550 bytes) 2024-09-15T00:10:10.738 INFO:teuthology.orchestra.run.smithi092.stdout:4952 trunc from 0xd62be00 to 0xe7a4600 2024-09-15T00:10:10.738 INFO:teuthology.orchestra.run.smithi092.stdout:4955 write 0x5322600 thru 0x532aa66 (0x8467 bytes) 2024-09-15T00:10:10.738 INFO:teuthology.orchestra.run.smithi092.stdout:4957 clone 33 order 25 su 8388608 sc 10 2024-09-15T00:10:10.835 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:10.833+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:10.841 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:10.841 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:10.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:10.936 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:10.935+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:10.992 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:10.992 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:10.992 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:10.992 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:10.992 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:11.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:11.042 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:11.040+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:11.093 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:10:11.138 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:11.136+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:11.153 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4070 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800667500000003, 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02702953, 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4070 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350998+0000", 2024-09-15T00:10:11.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.350999+0000", 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351003+0000", 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351007+0000", 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6820000000000001e-06 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351010+0000", 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7269999999999999e-06 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377929+0000", 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026919887999999999 2024-09-15T00:10:11.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.377969+0000", 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9909e-05 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378028+0000", 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8578000000000001e-05 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4071 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800652620000001, 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027136008, 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4071 2024-09-15T00:10:11.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351013+0000", 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351014+0000", 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9100000000000004e-07 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351015+0000", 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7039999999999999e-06 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351020+0000", 2024-09-15T00:10:11.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8450000000000002e-06 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351023+0000", 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378053+0000", 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027029610999999999 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378094+0000", 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1826999999999998e-05 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378149+0000", 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4815000000000003e-05 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.163 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4072 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800627941000002, 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027223404999999999, 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4072 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351038+0000", 2024-09-15T00:10:11.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351040+0000", 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0229999999999999e-06 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351046+0000", 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351051+0000", 2024-09-15T00:10:11.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7530000000000001e-06 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378171+0000", 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027120590999999999 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378209+0000", 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7308999999999998e-05 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378261+0000", 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2753999999999997e-05 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4074 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800609045999998, 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027334509999999999, 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4074 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351057+0000", 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9200000000000002e-07 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351059+0000", 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.012e-06 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351065+0000", 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2129999999999996e-06 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351070+0000", 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8470000000000003e-06 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378285+0000", 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027214449000000002 2024-09-15T00:10:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378334+0000", 2024-09-15T00:10:11.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9651000000000003e-05 2024-09-15T00:10:11.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378391+0000", 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7046000000000002e-05 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4075 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800590174999996, 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027546923000000001, 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.173 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4075 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351076+0000", 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351078+0000", 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.905e-06 2024-09-15T00:10:11.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351085+0000", 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351090+0000", 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7369999999999997e-06 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378423+0000", 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027333051000000001 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378523+0000", 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9959999999999998e-05 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378623+0000", 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9962000000000003e-05 2024-09-15T00:10:11.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4077 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800567491000002, 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027660619000000001, 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.178 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4077 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351098+0000", 2024-09-15T00:10:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351099+0000", 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351101+0000", 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351107+0000", 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7980000000000002e-06 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351111+0000", 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6399999999999996e-06 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378657+0000", 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027545463999999999 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378702+0000", 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5809999999999997e-05 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378759+0000", 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4078 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800539491000002, 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027808405000000001, 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4078 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.183 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351126+0000", 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351127+0000", 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7999999999999996e-07 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351129+0000", 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351135+0000", 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3820000000000001e-06 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351141+0000", 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2259999999999996e-06 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378785+0000", 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027644592999999999 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378855+0000", 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0276000000000003e-05 2024-09-15T00:10:11.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378935+0000", 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9361000000000001e-05 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4079 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800517853999999, 2024-09-15T00:10:11.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027913778, 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4079 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:11.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351148+0000", 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351150+0000", 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9010000000000001e-06 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351157+0000", 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8199999999999999e-06 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351162+0000", 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.378958+0000", 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027795786999999999 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379004+0000", 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6124000000000001e-05 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379062+0000", 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7630999999999998e-05 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.191 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4081 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800494172999997, 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016658, 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4081 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351172+0000", 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3500000000000002e-07 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351174+0000", 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351181+0000", 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9070000000000001e-06 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351186+0000", 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.651e-06 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379085+0000", 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027898757999999999 2024-09-15T00:10:11.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379130+0000", 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5126000000000001e-05 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379188+0000", 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8548999999999998e-05 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.196 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4082 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800474119, 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028118457999999999, 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.197 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4082 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351192+0000", 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351194+0000", 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7519999999999999e-06 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351200+0000", 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.801e-06 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351205+0000", 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3859999999999998e-06 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379213+0000", 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028007539000000001 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379254+0000", 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1817999999999998e-05 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379310+0000", 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5850000000000002e-05 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4083 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800452399000001, 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028218838999999999, 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4083 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.202 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351213+0000", 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351214+0000", 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351216+0000", 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351222+0000", 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8279999999999999e-06 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351226+0000", 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379333+0000", 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028106740000000002 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379376+0000", 2024-09-15T00:10:11.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3581000000000002e-05 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379432+0000", 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5868000000000001e-05 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4085 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:11.206 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800423895000002, 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311823, 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4085 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:11.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351242+0000", 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0699999999999998e-07 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351244+0000", 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.782e-06 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351248+0000", 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6289999999999999e-06 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351250+0000", 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379457+0000", 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028206479 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379499+0000", 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1637000000000001e-05 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379554+0000", 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5248999999999999e-05 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4086 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800412053999999, 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02842368, 2024-09-15T00:10:11.211 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4086 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:11.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351254+0000", 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351259+0000", 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9520000000000002e-06 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351262+0000", 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351265+0000", 2024-09-15T00:10:11.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6869999999999999e-06 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379576+0000", 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028311228000000001 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379619+0000", 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2595000000000001e-05 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379678+0000", 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8671000000000001e-05 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4087 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800397699000001, 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523903999999999, 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4087 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351268+0000", 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351269+0000", 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351270+0000", 2024-09-15T00:10:11.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1090000000000001e-06 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351273+0000", 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9189999999999999e-06 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351275+0000", 2024-09-15T00:10:11.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:11.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379701+0000", 2024-09-15T00:10:11.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028425813000000001 2024-09-15T00:10:11.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379742+0000", 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0769000000000002e-05 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379792+0000", 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0370000000000001e-05 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4088 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800381565000002, 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028621284, 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4088 2024-09-15T00:10:11.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351284+0000", 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351285+0000", 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351286+0000", 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.186e-06 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351289+0000", 2024-09-15T00:10:11.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351292+0000", 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7300000000000001e-06 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379815+0000", 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028523211 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379857+0000", 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1807999999999997e-05 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379906+0000", 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8594999999999998e-05 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.226 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4090 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800370479999998, 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028748587999999999, 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4090 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351295+0000", 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351296+0000", 2024-09-15T00:10:11.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.05e-07 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351297+0000", 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1060000000000001e-06 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351300+0000", 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9110000000000002e-06 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351302+0000", 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5509999999999998e-06 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379947+0000", 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028645177000000001 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.379991+0000", 2024-09-15T00:10:11.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3680999999999997e-05 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380044+0000", 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2957000000000001e-05 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4091 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:11.231 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800360925, 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028864198000000001, 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4091 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:11.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351305+0000", 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0599999999999999e-07 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351306+0000", 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351309+0000", 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351312+0000", 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-15T00:10:11.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380069+0000", 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028757933999999999 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380113+0000", 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3810000000000002e-05 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380169+0000", 2024-09-15T00:10:11.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5887999999999998e-05 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4092 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800351567, 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028973819000000001, 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4092 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351314+0000", 2024-09-15T00:10:11.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351315+0000", 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9999999999999999e-07 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351316+0000", 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.95e-07 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351318+0000", 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8629999999999999e-06 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351321+0000", 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.813e-06 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380195+0000", 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028873868 2024-09-15T00:10:11.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380238+0000", 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2883999999999998e-05 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380288+0000", 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0096000000000003e-05 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4094 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800341948000003, 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02907964, 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4094 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351324+0000", 2024-09-15T00:10:11.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351325+0000", 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.054e-06 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351328+0000", 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351331+0000", 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4679999999999999e-06 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380312+0000", 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028981587999999999 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380355+0000", 2024-09-15T00:10:11.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2688000000000002e-05 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380404+0000", 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8767999999999998e-05 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:4095 3.15 3:ae5153a6:::rbd_data.11296e862a2c.000000000000003f:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.800332560999998, 2024-09-15T00:10:11.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029185068000000002, 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4095 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:10:11.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351333+0000", 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351334+0000", 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.04e-07 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351335+0000", 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0300000000000001e-06 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351339+0000", 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.351344+0000", 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0560000000000004e-06 2024-09-15T00:10:11.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380428+0000", 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02908444 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380470+0000", 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1338000000000003e-05 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.380518+0000", 2024-09-15T00:10:11.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8724999999999999e-05 2024-09-15T00:10:11.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:11.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:11.250 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:11.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:11.252 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:11.250+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:11.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:11.352 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:11.350+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:11.400 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:11.400 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:11.400 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:11.400 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:11.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:11.453 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:11.450+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:11.454 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:11.471 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:11.472 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:11.489 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:11.489 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:11.505 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:11.505 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:11.522 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:11.522 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:11.539 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:11.539 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:11.541 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:11.542 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:11.553 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:11.551+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:11.555 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:11.555 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:11.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:11.607 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-15T00:10:11.654 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:11.652+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:11.689 INFO:teuthology.orchestra.run.smithi092.stdout:4960 punch from 0x45b8800 to 0x45b9400, (0xc00 bytes) 2024-09-15T00:10:11.697 INFO:teuthology.orchestra.run.smithi092.stdout:4963 punch from 0x466200 to 0x46d200, (0x7000 bytes) 2024-09-15T00:10:11.713 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:11.713 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:11.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:11.723 INFO:teuthology.orchestra.run.smithi092.stdout:4964 read 0x291ac00 thru 0x291d960 (0x2d61 bytes) 2024-09-15T00:10:11.723 INFO:teuthology.orchestra.run.smithi092.stdout:4966 write 0xd8e8e00 thru 0xd8f494d (0xbb4e bytes) 2024-09-15T00:10:11.724 INFO:teuthology.orchestra.run.smithi092.stdout:4967 trunc from 0xe7a4600 to 0xe5e1000 2024-09-15T00:10:11.739 INFO:teuthology.orchestra.run.smithi092.stdout:4968 punch from 0x1c51600 to 0x1c56200, (0x4c00 bytes) 2024-09-15T00:10:11.743 INFO:teuthology.orchestra.run.smithi092.stdout:4969 read 0xc37ca00 thru 0xc38b2f9 (0xe8fa bytes) 2024-09-15T00:10:11.743 INFO:teuthology.orchestra.run.smithi092.stdout:4970 read 0x2e2ee00 thru 0x2e30ea5 (0x20a6 bytes) 2024-09-15T00:10:11.743 INFO:teuthology.orchestra.run.smithi092.stdout:4973 write 0x4f89e00 thru 0x4f93158 (0x9359 bytes) 2024-09-15T00:10:11.756 INFO:teuthology.orchestra.run.smithi092.stdout:4976 trunc from 0xe5e1000 to 0x32c9c00 2024-09-15T00:10:11.765 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:11.764+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:11.833 INFO:teuthology.orchestra.run.smithi092.stdout:4977 write 0xc7fc200 thru 0xc8085cd (0xc3ce bytes) 2024-09-15T00:10:11.835 INFO:teuthology.orchestra.run.smithi092.stdout:4979 write 0x5d1d000 thru 0x5d28962 (0xb963 bytes) 2024-09-15T00:10:11.835 INFO:teuthology.orchestra.run.smithi092.stdout:4980 read 0xa49e000 thru 0xa4a1e54 (0x3e55 bytes) 2024-09-15T00:10:11.836 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:11.836 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:11.839 INFO:teuthology.orchestra.run.smithi092.stdout:4984 trunc from 0xc808600 to 0x9795a00 2024-09-15T00:10:11.846 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:11.859 INFO:teuthology.orchestra.run.smithi092.stdout:4986 write 0x1e87000 thru 0x1e8dae3 (0x6ae4 bytes) 2024-09-15T00:10:11.860 INFO:teuthology.orchestra.run.smithi092.stdout:4989 write 0x1741000 thru 0x17442d2 (0x32d3 bytes) 2024-09-15T00:10:11.860 INFO:teuthology.orchestra.run.smithi092.stdout:4990 read 0x2027e00 thru 0x202b8a6 (0x3aa7 bytes) 2024-09-15T00:10:11.860 INFO:teuthology.orchestra.run.smithi092.stdout:4991 punch from 0xf4f600 to 0xf57a00, (0x8400 bytes) 2024-09-15T00:10:11.861 INFO:teuthology.orchestra.run.smithi092.stdout:4992 punch from 0x1863600 to 0x1866200, (0x2c00 bytes) 2024-09-15T00:10:11.870 INFO:teuthology.orchestra.run.smithi092.stdout:4995 write 0xa17a400 thru 0xa17cb37 (0x2738 bytes) 2024-09-15T00:10:11.873 INFO:teuthology.orchestra.run.smithi092.stdout:4996 read 0x9349800 thru 0x935964e (0xfe4f bytes) 2024-09-15T00:10:11.874 INFO:teuthology.orchestra.run.smithi092.stdout:4997 write 0x94ad200 thru 0x94affcf (0x2dd0 bytes) 2024-09-15T00:10:11.874 INFO:teuthology.orchestra.run.smithi092.stdout:4999 read 0x3ff8800 thru 0x40008ee (0x80ef bytes) 2024-09-15T00:10:11.874 INFO:teuthology.orchestra.run.smithi092.stdout:5001 read 0x9296c00 thru 0x929720e (0x60f bytes) 2024-09-15T00:10:11.874 INFO:teuthology.orchestra.run.smithi092.stdout:5003 read 0x18ce000 thru 0x18d922a (0xb22b bytes) 2024-09-15T00:10:11.874 INFO:teuthology.orchestra.run.smithi092.stdout:5004 read 0x368d600 thru 0x369d184 (0xfb85 bytes) 2024-09-15T00:10:11.875 INFO:teuthology.orchestra.run.smithi092.stdout:5005 punch from 0x4078a00 to 0x407c600, (0x3c00 bytes) 2024-09-15T00:10:11.875 INFO:teuthology.orchestra.run.smithi092.stdout:5006 punch from 0x5a28000 to 0x5a30400, (0x8400 bytes) 2024-09-15T00:10:11.875 INFO:teuthology.orchestra.run.smithi092.stdout:5007 read 0xd17800 thru 0xd1f3f0 (0x7bf1 bytes) 2024-09-15T00:10:11.876 INFO:teuthology.orchestra.run.smithi092.stdout:5008 punch from 0x281e800 to 0x282b000, (0xc800 bytes) 2024-09-15T00:10:11.882 INFO:teuthology.orchestra.run.smithi092.stdout:5009 trunc from 0xa17cc00 to 0x4e28c00 2024-09-15T00:10:11.887 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:11.885+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:11.894 INFO:teuthology.orchestra.run.smithi092.stdout:5012 read 0x4859800 thru 0x4861b89 (0x838a bytes) 2024-09-15T00:10:11.914 INFO:teuthology.orchestra.run.smithi092.stdout:5013 write 0xe406200 thru 0xe406938 (0x739 bytes) 2024-09-15T00:10:11.916 INFO:teuthology.orchestra.run.smithi092.stdout:5014 punch from 0x73f4600 to 0x73f4a00, (0x400 bytes) 2024-09-15T00:10:11.920 INFO:teuthology.orchestra.run.smithi092.stdout:5016 write 0x1471c00 thru 0x148023f (0xe640 bytes) 2024-09-15T00:10:11.930 INFO:teuthology.orchestra.run.smithi092.stdout:5019 trunc from 0xe406a00 to 0x6271800 2024-09-15T00:10:11.947 INFO:teuthology.orchestra.run.smithi092.stdout:5020 read 0x5432c00 thru 0x5439df0 (0x71f1 bytes) 2024-09-15T00:10:11.953 INFO:teuthology.orchestra.run.smithi092.stdout:5021 punch from 0x1790600 to 0x1791200, (0xc00 bytes) 2024-09-15T00:10:11.955 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:11.954+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:11.971 INFO:teuthology.orchestra.run.smithi092.stdout:5023 write 0x9e97000 thru 0x9e99b50 (0x2b51 bytes) 2024-09-15T00:10:11.983 INFO:teuthology.orchestra.run.smithi092.stdout:5024 trunc from 0x9e99c00 to 0x397800 2024-09-15T00:10:11.995 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:11.995 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:12.005 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:12.032 INFO:teuthology.orchestra.run.smithi092.stdout:5027 punch from 0x40600 to 0x45200, (0x4c00 bytes) 2024-09-15T00:10:12.033 INFO:teuthology.orchestra.run.smithi092.stdout:5028 read 0x36c600 thru 0x36ef9e (0x299f bytes) 2024-09-15T00:10:12.054 INFO:teuthology.orchestra.run.smithi092.stdout:5029 write 0x8cd1e00 thru 0x8cd7433 (0x5634 bytes) 2024-09-15T00:10:12.056 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:12.054+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:12.058 INFO:teuthology.orchestra.run.smithi092.stdout:5030 trunc from 0x8cd7600 to 0xb805600 2024-09-15T00:10:12.069 INFO:teuthology.orchestra.run.smithi092.stdout:5031 punch from 0x1414600 to 0x1419400, (0x4e00 bytes) 2024-09-15T00:10:12.076 INFO:teuthology.orchestra.run.smithi092.stdout:5033 write 0x2634200 thru 0x263ca80 (0x8881 bytes) 2024-09-15T00:10:12.090 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:10:12.094 INFO:teuthology.orchestra.run.smithi092.stdout:5034 write 0xdc69a00 thru 0xdc78086 (0xe687 bytes) 2024-09-15T00:10:12.095 INFO:teuthology.orchestra.run.smithi092.stdout:5038 punch from 0xb91f400 to 0xb92ba00, (0xc600 bytes) 2024-09-15T00:10:12.095 INFO:teuthology.orchestra.run.smithi092.stdout:5039 punch from 0x87ec400 to 0x87f0000, (0x3c00 bytes) 2024-09-15T00:10:12.095 INFO:teuthology.orchestra.run.smithi092.stdout:5041 write 0x529fa00 thru 0x52a09a3 (0xfa4 bytes) 2024-09-15T00:10:12.096 INFO:teuthology.orchestra.run.smithi092.stdout:5042 trunc from 0xdc78200 to 0xdc61e00 2024-09-15T00:10:12.108 INFO:teuthology.orchestra.run.smithi092.stdout:5044 write 0xa734a00 thru 0xa738ca7 (0x42a8 bytes) 2024-09-15T00:10:12.117 INFO:teuthology.orchestra.run.smithi092.stdout:5045 read 0xbb9a200 thru 0xbba5a6c (0xb86d bytes) 2024-09-15T00:10:12.117 INFO:teuthology.orchestra.run.smithi092.stdout:5046 write 0xc1f0200 thru 0xc1f9d86 (0x9b87 bytes) 2024-09-15T00:10:12.117 INFO:teuthology.orchestra.run.smithi092.stdout:5049 write 0x7d5e600 thru 0x7d68680 (0xa081 bytes) 2024-09-15T00:10:12.117 INFO:teuthology.orchestra.run.smithi092.stdout:5050 read 0xb423c00 thru 0xb4251da (0x15db bytes) 2024-09-15T00:10:12.119 INFO:teuthology.orchestra.run.smithi092.stdout:5052 trunc from 0xdc61e00 to 0xee2ae00 2024-09-15T00:10:12.129 INFO:teuthology.orchestra.run.smithi092.stdout:5053 write 0x21fe00 thru 0x22f971 (0xfb72 bytes) 2024-09-15T00:10:12.132 INFO:teuthology.orchestra.run.smithi092.stdout:5054 write 0x5362a00 thru 0x53632fc (0x8fd bytes) 2024-09-15T00:10:12.132 INFO:teuthology.orchestra.run.smithi092.stdout:5057 read 0xc807800 thru 0xc80d698 (0x5e99 bytes) 2024-09-15T00:10:12.145 INFO:teuthology.orchestra.run.smithi092.stdout:5066 trunc from 0xee2ae00 to 0x4403600 2024-09-15T00:10:12.156 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:12.155+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:12.157 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:12.157 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:12.165 INFO:teuthology.orchestra.run.smithi092.stdout:5067 punch from 0xb0f800 to 0xb14800, (0x5000 bytes) 2024-09-15T00:10:12.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:12.178 INFO:teuthology.orchestra.run.smithi092.stdout:5068 trunc from 0x4403600 to 0xd43e800 2024-09-15T00:10:12.187 INFO:teuthology.orchestra.run.smithi092.stdout:5070 read 0x4197a00 thru 0x41a30c2 (0xb6c3 bytes) 2024-09-15T00:10:12.188 INFO:teuthology.orchestra.run.smithi092.stdout:5071 trunc from 0xd43e800 to 0xd45fa00 2024-09-15T00:10:12.212 INFO:teuthology.orchestra.run.smithi092.stdout:5072 write 0xe08fc00 thru 0xe09a836 (0xac37 bytes) 2024-09-15T00:10:12.220 INFO:teuthology.orchestra.run.smithi092.stdout:5073 punch from 0x1545000 to 0x1545400, (0x400 bytes) 2024-09-15T00:10:12.221 INFO:teuthology.orchestra.run.smithi092.stdout:5077 write 0x8f73c00 thru 0x8f73d05 (0x106 bytes) 2024-09-15T00:10:12.221 INFO:teuthology.orchestra.run.smithi092.stdout:5081 read 0xce7dc00 thru 0xce86b2a (0x8f2b bytes) 2024-09-15T00:10:12.234 INFO:teuthology.orchestra.run.smithi092.stdout:5082 write 0xeaf9600 thru 0xeb051ed (0xbbee bytes) 2024-09-15T00:10:12.235 INFO:teuthology.orchestra.run.smithi092.stdout:5083 read 0x3399600 thru 0x33a46a8 (0xb0a9 bytes) 2024-09-15T00:10:12.239 INFO:teuthology.orchestra.run.smithi092.stdout:5084 read 0xab3c000 thru 0xab3db4d (0x1b4e bytes) 2024-09-15T00:10:12.240 INFO:teuthology.orchestra.run.smithi092.stdout:5087 write 0x6d76a00 thru 0x6d83160 (0xc761 bytes) 2024-09-15T00:10:12.240 INFO:teuthology.orchestra.run.smithi092.stdout:5089 read 0xe2c4600 thru 0xe2ca17a (0x5b7b bytes) 2024-09-15T00:10:12.240 INFO:teuthology.orchestra.run.smithi092.stdout:5090 write 0xc8e8c00 thru 0xc8f80a1 (0xf4a2 bytes) 2024-09-15T00:10:12.241 INFO:teuthology.orchestra.run.smithi092.stdout:5092 write 0xcde3200 thru 0xcdf199e (0xe79f bytes) 2024-09-15T00:10:12.241 INFO:teuthology.orchestra.run.smithi092.stdout:5096 write 0xc2e7600 thru 0xc2e7ffe (0x9ff bytes) 2024-09-15T00:10:12.244 INFO:teuthology.orchestra.run.smithi092.stdout:5097 trunc from 0xeb05200 to 0xc989400 2024-09-15T00:10:12.258 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:12.256+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:12.259 INFO:teuthology.orchestra.run.smithi092.stdout:5099 read 0x2009600 thru 0x201528f (0xbc90 bytes) 2024-09-15T00:10:12.261 INFO:teuthology.orchestra.run.smithi092.stdout:5100 write 0x56c7e00 thru 0x56d2b0b (0xad0c bytes) 2024-09-15T00:10:12.262 INFO:teuthology.orchestra.run.smithi092.stdout:5102 punch from 0x9be9400 to 0x9bf2800, (0x9400 bytes) 2024-09-15T00:10:12.262 INFO:teuthology.orchestra.run.smithi092.stdout:5103 read 0x714a800 thru 0x714e62d (0x3e2e bytes) 2024-09-15T00:10:12.262 INFO:teuthology.orchestra.run.smithi092.stdout:5104 write 0x1fd1000 thru 0x1fdec52 (0xdc53 bytes) 2024-09-15T00:10:12.264 INFO:teuthology.orchestra.run.smithi092.stdout:5105 write 0x8729e00 thru 0x8738e5c (0xf05d bytes) 2024-09-15T00:10:12.265 INFO:teuthology.orchestra.run.smithi092.stdout:5107 read 0x420b200 thru 0x4212bfa (0x79fb bytes) 2024-09-15T00:10:12.265 INFO:teuthology.orchestra.run.smithi092.stdout:5109 write 0x8f1b200 thru 0x8f20865 (0x5666 bytes) 2024-09-15T00:10:12.266 INFO:teuthology.orchestra.run.smithi092.stdout:5110 read 0x451ea00 thru 0x4526b36 (0x8137 bytes) 2024-09-15T00:10:12.266 INFO:teuthology.orchestra.run.smithi092.stdout:5111 read 0x82ab600 thru 0x82ae2ef (0x2cf0 bytes) 2024-09-15T00:10:12.266 INFO:teuthology.orchestra.run.smithi092.stdout:5112 write 0x1c6d400 thru 0x1c726a4 (0x52a5 bytes) 2024-09-15T00:10:12.272 INFO:teuthology.orchestra.run.smithi092.stdout:5113 trunc from 0xc989400 to 0x7c42c00 2024-09-15T00:10:12.288 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:12.288 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:12.291 INFO:teuthology.orchestra.run.smithi092.stdout:5116 trunc from 0x7c42c00 to 0x8f3a000 2024-09-15T00:10:12.298 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:12.301 INFO:teuthology.orchestra.run.smithi092.stdout:5118 write 0x6550c00 thru 0x655e73c (0xdb3d bytes) 2024-09-15T00:10:12.307 INFO:teuthology.orchestra.run.smithi092.stdout:5124 trunc from 0x8f3a000 to 0xd5b4600 2024-09-15T00:10:12.318 INFO:teuthology.orchestra.run.smithi092.stdout:5126 write 0xd420a00 thru 0xd421b98 (0x1199 bytes) 2024-09-15T00:10:12.320 INFO:teuthology.orchestra.run.smithi092.stdout:5128 read 0x2bae800 thru 0x2bb5e35 (0x7636 bytes) 2024-09-15T00:10:12.321 INFO:teuthology.orchestra.run.smithi092.stdout:5129 read 0x396400 thru 0x3981bf (0x1dc0 bytes) 2024-09-15T00:10:12.325 INFO:teuthology.orchestra.run.smithi092.stdout:5132 trunc from 0xd5b4600 to 0xaf6fe00 2024-09-15T00:10:12.336 INFO:teuthology.orchestra.run.smithi092.stdout:5133 read 0x3d82200 thru 0x3d9081e (0xe61f bytes) 2024-09-15T00:10:12.337 INFO:teuthology.orchestra.run.smithi092.stdout:5137 trunc from 0xaf6fe00 to 0xb3bb400 2024-09-15T00:10:12.345 INFO:teuthology.orchestra.run.smithi092.stdout:5138 read 0x183400 thru 0x186f2b (0x3b2c bytes) 2024-09-15T00:10:12.347 INFO:teuthology.orchestra.run.smithi092.stdout:5139 punch from 0x7255000 to 0x7258e00, (0x3e00 bytes) 2024-09-15T00:10:12.348 INFO:teuthology.orchestra.run.smithi092.stdout:5140 write 0x8fe0800 thru 0x8fe200a (0x180b bytes) 2024-09-15T00:10:12.348 INFO:teuthology.orchestra.run.smithi092.stdout:5141 read 0x58efc00 thru 0x58f8741 (0x8b42 bytes) 2024-09-15T00:10:12.348 INFO:teuthology.orchestra.run.smithi092.stdout:5143 read 0x60a1800 thru 0x60a6593 (0x4d94 bytes) 2024-09-15T00:10:12.349 INFO:teuthology.orchestra.run.smithi092.stdout:5144 punch from 0x77d5600 to 0x77e4000, (0xea00 bytes) 2024-09-15T00:10:12.349 INFO:teuthology.orchestra.run.smithi092.stdout:5149 punch from 0x917b800 to 0x917e800, (0x3000 bytes) 2024-09-15T00:10:12.349 INFO:teuthology.orchestra.run.smithi092.stdout:5154 read 0x59c7200 thru 0x59d5514 (0xe315 bytes) 2024-09-15T00:10:12.349 INFO:teuthology.orchestra.run.smithi092.stdout:5156 read 0x168ac00 thru 0x1690a8d (0x5e8e bytes) 2024-09-15T00:10:12.350 INFO:teuthology.orchestra.run.smithi092.stdout:5159 punch from 0xd53200 to 0xd58200, (0x5000 bytes) 2024-09-15T00:10:12.350 INFO:teuthology.orchestra.run.smithi092.stdout:5160 read 0x7ddca00 thru 0x7de4cf0 (0x82f1 bytes) 2024-09-15T00:10:12.351 INFO:teuthology.orchestra.run.smithi092.stdout:5161 punch from 0xa41a400 to 0xa425800, (0xb400 bytes) 2024-09-15T00:10:12.351 INFO:teuthology.orchestra.run.smithi092.stdout:5163 punch from 0x79aea00 to 0x79bc000, (0xd600 bytes) 2024-09-15T00:10:12.351 INFO:teuthology.orchestra.run.smithi092.stdout:5164 read 0x436de00 thru 0x437cff5 (0xf1f6 bytes) 2024-09-15T00:10:12.358 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:12.357+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:12.362 INFO:teuthology.orchestra.run.smithi092.stdout:5166 write 0xd3f4a00 thru 0xd3fbb0c (0x710d bytes) 2024-09-15T00:10:12.363 INFO:teuthology.orchestra.run.smithi092.stdout:5168 write 0x7960000 thru 0x796557f (0x5580 bytes) 2024-09-15T00:10:12.363 INFO:teuthology.orchestra.run.smithi092.stdout:5169 punch from 0xa249c00 to 0xa24ba00, (0x1e00 bytes) 2024-09-15T00:10:12.372 INFO:teuthology.orchestra.run.smithi092.stdout:5170 trunc from 0xd3fbc00 to 0x64e3600 2024-09-15T00:10:12.387 INFO:teuthology.orchestra.run.smithi092.stdout:skipping zero length punch hole 2024-09-15T00:10:12.387 INFO:teuthology.orchestra.run.smithi092.stdout:5172 read 0x62cce00 thru 0x62d8620 (0xb821 bytes) 2024-09-15T00:10:12.388 INFO:teuthology.orchestra.run.smithi092.stdout:5173 read 0x20ce800 thru 0x20d5c1a (0x741b bytes) 2024-09-15T00:10:12.391 INFO:teuthology.orchestra.run.smithi092.stdout:5179 trunc from 0x64e3600 to 0x8294000 2024-09-15T00:10:12.414 INFO:teuthology.orchestra.run.smithi092.stdout:5180 write 0xd6da600 thru 0xd6dcb22 (0x2523 bytes) 2024-09-15T00:10:12.415 INFO:teuthology.orchestra.run.smithi092.stdout:5182 punch from 0xd67400 to 0xd72800, (0xb400 bytes) 2024-09-15T00:10:12.418 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:12.418 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:12.420 INFO:teuthology.orchestra.run.smithi092.stdout:5184 trunc from 0xd6dcc00 to 0x9f58200 2024-09-15T00:10:12.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:12.438 INFO:teuthology.orchestra.run.smithi092.stdout:5186 trunc from 0x9f58200 to 0x43c1600 2024-09-15T00:10:12.453 INFO:teuthology.orchestra.run.smithi092.stdout:5187 read 0x3c58e00 thru 0x3c629a7 (0x9ba8 bytes) 2024-09-15T00:10:12.454 INFO:teuthology.orchestra.run.smithi092.stdout:5189 write 0x381600 thru 0x3859a4 (0x43a5 bytes) 2024-09-15T00:10:12.457 INFO:teuthology.orchestra.run.smithi092.stdout:5194 punch from 0x963e00 to 0x96dc00, (0x9e00 bytes) 2024-09-15T00:10:12.457 INFO:teuthology.orchestra.run.smithi092.stdout:5195 read 0x227e000 thru 0x2288827 (0xa828 bytes) 2024-09-15T00:10:12.459 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:12.457+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:12.466 INFO:teuthology.orchestra.run.smithi092.stdout:5196 trunc from 0x43c1600 to 0xc109200 2024-09-15T00:10:12.476 INFO:teuthology.orchestra.run.smithi092.stdout:5197 trunc from 0xc109200 to 0xb64a800 2024-09-15T00:10:12.487 INFO:teuthology.orchestra.run.smithi092.stdout:5198 write 0x3b0da00 thru 0x3b1341e (0x5a1f bytes) 2024-09-15T00:10:12.488 INFO:teuthology.orchestra.run.smithi092.stdout:5201 read 0x519d600 thru 0x51ab600 (0xe001 bytes) 2024-09-15T00:10:12.490 INFO:teuthology.orchestra.run.smithi092.stdout:5204 read 0x6795000 thru 0x6799d49 (0x4d4a bytes) 2024-09-15T00:10:12.490 INFO:teuthology.orchestra.run.smithi092.stdout:5205 read 0x6825000 thru 0x6825421 (0x422 bytes) 2024-09-15T00:10:12.490 INFO:teuthology.orchestra.run.smithi092.stdout:5207 read 0x2f6ea00 thru 0x2f753a9 (0x69aa bytes) 2024-09-15T00:10:12.491 INFO:teuthology.orchestra.run.smithi092.stdout:5208 read 0x22e4800 thru 0x22e85dd (0x3dde bytes) 2024-09-15T00:10:12.491 INFO:teuthology.orchestra.run.smithi092.stdout:5210 read 0x525b800 thru 0x5260743 (0x4f44 bytes) 2024-09-15T00:10:12.505 INFO:teuthology.orchestra.run.smithi092.stdout:5211 write 0xe041400 thru 0xe041d2f (0x930 bytes) 2024-09-15T00:10:12.506 INFO:teuthology.orchestra.run.smithi092.stdout:5213 clone 34 order 19 su 4096 sc 10 2024-09-15T00:10:12.545 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:12.545 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:12.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:12.597 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:12.595+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:12.661 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:12.659+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:12.661 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:12.661 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:12.670 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:12.762 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:12.761+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:12.817 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:12.817 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:12.825 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:12.870 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:12.868+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:12.932 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:12.933 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:12.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:12.985 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:12.984+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:13.045 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:13.045 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:13.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:13.095 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:13.093+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:13.165 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone33 from 0xe041e00 (overlap 0x397800) to 0x0 2024-09-15T00:10:13.166 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:13.164+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:13.203 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:13.203 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:13.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:13.296 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:13.294+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:13.358 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:13.359 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:13.367 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:13.365+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:13.368 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:13.447 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-15T00:10:13.467 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:13.466+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:13.514 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:13.515 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:13.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:13.543 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:13.569 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:13.568+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:13.658 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:13.664 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:13.664 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:13.668 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:13.666+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:13.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:13.678 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:13.678 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:13.699 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:13.699 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:13.721 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:13.741 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:13.741 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:13.763 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:13.763 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:13.767 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:13.767 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:13.808 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:13.806+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:13.853 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3427 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563248469999998, 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027724631, 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3427 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287581+0000", 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287582+0000", 2024-09-15T00:10:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2599999999999998e-07 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287584+0000", 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8759999999999999e-06 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287589+0000", 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287594+0000", 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.357e-06 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315217+0000", 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027622785 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315257+0000", 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0228000000000002e-05 2024-09-15T00:10:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315306+0000", 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9228999999999999e-05 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3428 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563230464999997, 2024-09-15T00:10:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027928562000000001, 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3428 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287599+0000", 2024-09-15T00:10:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287600+0000", 2024-09-15T00:10:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287602+0000", 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287608+0000", 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287613+0000", 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8740000000000003e-06 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315346+0000", 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027733099000000001 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315431+0000", 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4350000000000004e-05 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315528+0000", 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7374000000000007e-05 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3429 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563211154999998, 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028028630999999998, 2024-09-15T00:10:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3429 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287619+0000", 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287621+0000", 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287627+0000", 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8440000000000003e-06 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287631+0000", 2024-09-15T00:10:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5009999999999998e-06 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315556+0000", 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027925241999999999 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315595+0000", 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8512e-05 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315647+0000", 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2318000000000003e-05 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3430 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 126976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563193081000001, 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028122114, 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3430 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287637+0000", 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287644+0000", 2024-09-15T00:10:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4219999999999997e-06 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287650+0000", 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0920000000000003e-06 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287654+0000", 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4279999999999998e-06 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315671+0000", 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028016592 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315709+0000", 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8231e-05 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315759+0000", 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9926999999999998e-05 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.872 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3431 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563169541000001, 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02831842, 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3431 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287660+0000", 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287661+0000", 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287663+0000", 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9570000000000001e-06 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287668+0000", 2024-09-15T00:10:13.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7679999999999997e-06 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287673+0000", 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5759999999999999e-06 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315799+0000", 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028125741999999999 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315891+0000", 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2701000000000006e-05 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.315979+0000", 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7287000000000006e-05 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3432 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563151157, 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028430624000000002, 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3432 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287679+0000", 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-15T00:10:13.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287681+0000", 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8530000000000001e-06 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287686+0000", 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4659999999999998e-06 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287691+0000", 2024-09-15T00:10:13.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2230000000000001e-06 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316006+0000", 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028315344999999999 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316054+0000", 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8004999999999999e-05 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316109+0000", 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5374999999999998e-05 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3433 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563133401000002, 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028528593000000001, 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.882 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3433 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287696+0000", 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287697+0000", 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287699+0000", 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8509999999999999e-06 2024-09-15T00:10:13.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287704+0000", 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287710+0000", 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316124+0000", 2024-09-15T00:10:13.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028414466999999999 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316171+0000", 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6783999999999998e-05 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316225+0000", 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3891999999999998e-05 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3434 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563114016, 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028632801999999999, 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3434 2024-09-15T00:10:13.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287716+0000", 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-07 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287718+0000", 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0200000000000001e-06 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287724+0000", 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8390000000000003e-06 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287729+0000", 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8389999999999997e-06 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316250+0000", 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028521135 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316295+0000", 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4540999999999999e-05 2024-09-15T00:10:13.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316349+0000", 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4083999999999999e-05 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3435 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563083339999999, 2024-09-15T00:10:13.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028723243999999998, 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3435 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287746+0000", 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287747+0000", 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4900000000000002e-07 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287748+0000", 2024-09-15T00:10:13.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4190000000000001e-06 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287752+0000", 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7900000000000001e-06 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287756+0000", 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1609999999999996e-06 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316375+0000", 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028618932 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316417+0000", 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2351000000000002e-05 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316470+0000", 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2342000000000001e-05 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3436 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563068291999997, 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028827854, 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3436 2024-09-15T00:10:13.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287762+0000", 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3200000000000001e-07 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287766+0000", 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287771+0000", 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5140000000000001e-06 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287776+0000", 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0790000000000002e-06 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316494+0000", 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028718144000000001 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316538+0000", 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4129999999999999e-05 2024-09-15T00:10:13.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316589+0000", 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1548999999999997e-05 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3437 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563050400000002, 2024-09-15T00:10:13.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028972188999999999, 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3437 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287779+0000", 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287780+0000", 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287782+0000", 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8419999999999999e-06 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287787+0000", 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3059999999999997e-06 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287791+0000", 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9920000000000001e-06 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316615+0000", 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028824487999999999 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316668+0000", 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2883999999999997e-05 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316752+0000", 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3308999999999998e-05 2024-09-15T00:10:13.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3438 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 192512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563033357999998, 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029124331999999999, 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3438 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287796+0000", 2024-09-15T00:10:13.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287797+0000", 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287799+0000", 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.849e-06 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287804+0000", 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7069999999999997e-06 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287808+0000", 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0720000000000001e-06 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316799+0000", 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028990241999999999 2024-09-15T00:10:13.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316845+0000", 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5837000000000002e-05 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316921+0000", 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6255999999999996e-05 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3439 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.563015819, 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029290665, 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3439 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287814+0000", 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-15T00:10:13.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287816+0000", 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6980000000000001e-06 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287824+0000", 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2500000000000006e-06 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287829+0000", 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4669999999999998e-06 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.316967+0000", 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029138319999999999 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317047+0000", 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0081999999999997e-05 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317105+0000", 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7497999999999997e-05 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3440 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.562994760000002, 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029395451999999999, 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.914 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3440 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287835+0000", 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287837+0000", 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-15T00:10:13.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287843+0000", 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6799999999999998e-06 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287848+0000", 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317131+0000", 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029283337999999999 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317176+0000", 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5176999999999998e-05 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317231+0000", 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4120999999999999e-05 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.918 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3441 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.562976696, 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029496321999999998, 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.919 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3441 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287853+0000", 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287855+0000", 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7099999999999999e-06 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287861+0000", 2024-09-15T00:10:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9689999999999999e-06 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287866+0000", 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5369999999999999e-06 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317252+0000", 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029386051999999999 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317297+0000", 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5164999999999999e-05 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317349+0000", 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2534000000000002e-05 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3442 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.923 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.562957429999997, 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029594697999999999, 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3442 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287872+0000", 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287873+0000", 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-15T00:10:13.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287879+0000", 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287885+0000", 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.755e-06 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287890+0000", 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317375+0000", 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029485457 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317416+0000", 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1217999999999997e-05 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317467+0000", 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0819999999999998e-05 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3443 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.562934091000002, 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029684676, 2024-09-15T00:10:13.928 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3443 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287896+0000", 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287898+0000", 2024-09-15T00:10:13.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287904+0000", 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8860000000000001e-06 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287908+0000", 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5650000000000003e-06 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317488+0000", 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029579437 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317530+0000", 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1900999999999997e-05 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317580+0000", 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0603000000000003e-05 2024-09-15T00:10:13.932 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3444 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 241664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.562908872999998, 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029772823, 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3444 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287921+0000", 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287931+0000", 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0008e-05 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287938+0000", 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6939999999999998e-06 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287943+0000", 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1070000000000004e-06 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317603+0000", 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029659913999999999 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317646+0000", 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2367999999999998e-05 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317694+0000", 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8201999999999997e-05 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.937 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3445 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.562879535, 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029863565000000002, 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3445 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287950+0000", 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287951+0000", 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287953+0000", 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.765e-06 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287958+0000", 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7899999999999996e-06 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287963+0000", 2024-09-15T00:10:13.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.634e-06 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317714+0000", 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029751353000000001 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317755+0000", 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0342000000000001e-05 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317814+0000", 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9188999999999999e-05 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4622.0:3446 3.9 3:97ada1ed:::rbd_data.11296e862a2c.0000000000000022:c [read 258048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.562861237, 2024-09-15T00:10:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029967884, 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4622", 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/4151905923", 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3446 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287969+0000", 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-15T00:10:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287971+0000", 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287977+0000", 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4330000000000003e-06 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.287981+0000", 2024-09-15T00:10:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3989999999999997e-06 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317837+0000", 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.029855942 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317878+0000", 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1467000000000001e-05 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:20.317936+0000", 2024-09-15T00:10:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8100999999999998e-05 2024-09-15T00:10:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:13.947 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:13.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:13.949 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:13.947+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:14.049 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:14.048+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:14.091 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:14.092 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:14.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:14.106 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:10:14.150 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:14.148+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:14.168 INFO:teuthology.orchestra.run.smithi092.stdout:5214 write 0x395400 thru 0x399811 (0x4412 bytes) 2024-09-15T00:10:14.179 INFO:teuthology.orchestra.run.smithi092.stdout:5219 trunc from 0xe041e00 to 0x4e11a00 2024-09-15T00:10:14.204 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:14.204 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:14.204 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:14.205 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:14.205 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:14.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:14.255 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:14.253+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:14.319 INFO:teuthology.orchestra.run.smithi092.stdout:5220 punch from 0x25cdc00 to 0x25d8a00, (0xae00 bytes) 2024-09-15T00:10:14.352 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:14.350+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:14.376 INFO:teuthology.orchestra.run.smithi092.stdout:5223 read 0x29fca00 thru 0x29fdaa3 (0x10a4 bytes) 2024-09-15T00:10:14.377 INFO:teuthology.orchestra.run.smithi092.stdout:5224 write 0x3017600 thru 0x30245c0 (0xcfc1 bytes) 2024-09-15T00:10:14.391 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.505702972000002, 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.505664342000003, 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.505638404999999, 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.505622336999998, 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.505481242000002, 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1148 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 827392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:10:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.504709487, 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045202784000000003, 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1148 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:10:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883797+0000", 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883798+0000", 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3399999999999999e-07 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883801+0000", 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6979999999999998e-06 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883811+0000", 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0127e-05 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883821+0000", 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0474e-05 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928861+0000", 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045040069000000002 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928905+0000", 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3439999999999997e-05 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929000+0000", 2024-09-15T00:10:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5441999999999999e-05 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.504624456000002, 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.504589439999997, 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.504563836999999, 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.504389570999997, 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.504158431999997, 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.503346487999998, 2024-09-15T00:10:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.503266748000001, 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.503061723000002, 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.502850189999997, 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.500294183000001, 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.471 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:14.472 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.498311076999997, 2024-09-15T00:10:14.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:14.472 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.472 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.472 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:14.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:14.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:14.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:14.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:14.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.498291506000001, 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:14.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:14.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11360 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.22756526, 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045212219999999997, 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:14.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11360 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160942+0000", 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.46e-07 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160944+0000", 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0420000000000001e-06 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160949+0000", 2024-09-15T00:10:14.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2320000000000001e-06 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160953+0000", 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0470000000000004e-06 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.205971+0000", 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045017313000000003 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206046+0000", 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5457000000000006e-05 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206154+0000", 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010778299999999999 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.486 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:14.487 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:14.487 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.227548656, 2024-09-15T00:10:14.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:14.487 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:14.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:14.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:14.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:14.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:14.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:14.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:14.491 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:14.493 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:14.494 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:14.490+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:14.586 INFO:teuthology.orchestra.run.smithi092.stdout:5225 write 0x90e3200 thru 0x90ed484 (0xa285 bytes) 2024-09-15T00:10:14.594 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:14.592+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:14.611 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5568.0:4081 3.0 3:064a5c05:::rbd_data.1129696a4676.0000000000000000:24 [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:14.609921+0000", 2024-09-15T00:10:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.4943000000000002e-05, 2024-09-15T00:10:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000130402, 2024-09-15T00:10:14.611 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5568", 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4081 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609921+0000", 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609921+0000", 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609922+0000", 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.16e-07 2024-09-15T00:10:14.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609926+0000", 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3479999999999997e-06 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609939+0000", 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.255e-05 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609946+0000", 2024-09-15T00:10:14.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3220000000000004e-06 2024-09-15T00:10:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609999+0000", 2024-09-15T00:10:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3452000000000003e-05 2024-09-15T00:10:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5568.0:4083 3.0 3.a03a5260 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:14.609959+0000", 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.6626000000000003e-05, 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000166188, 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5568", 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4083 2024-09-15T00:10:14.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609959+0000", 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609959+0000", 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609960+0000", 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6800000000000001e-07 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609961+0000", 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9400000000000001e-06 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609967+0000", 2024-09-15T00:10:14.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3809999999999998e-06 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609970+0000", 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2590000000000001e-06 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5568.0:4084 3.0 3.a03a5260 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:14.609975+0000", 2024-09-15T00:10:14.619 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.1140000000000003e-05, 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00021162700000000001, 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5568", 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 4084 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:14.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609975+0000", 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609975+0000", 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609975+0000", 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609976+0000", 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3e-06 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609980+0000", 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3189999999999999e-06 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:14.609982+0000", 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.616e-06 2024-09-15T00:10:14.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:14.623 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 3 2024-09-15T00:10:14.623 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:14.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:14.625 INFO:teuthology.orchestra.run.smithi092.stdout:5226 write 0xdd9e400 thru 0xdda563b (0x723c bytes) 2024-09-15T00:10:14.625 INFO:teuthology.orchestra.run.smithi092.stdout:5229 punch from 0x5b86800 to 0x5b86a00, (0x200 bytes) 2024-09-15T00:10:14.626 INFO:teuthology.orchestra.run.smithi092.stdout:5233 read 0xc4cba00 thru 0xc4cdc2e (0x222f bytes) 2024-09-15T00:10:14.626 INFO:teuthology.orchestra.run.smithi092.stdout:5238 read 0x5ffbc00 thru 0x600b517 (0xf918 bytes) 2024-09-15T00:10:14.626 INFO:teuthology.orchestra.run.smithi092.stdout:5239 trunc from 0xdda5800 to 0xd810000 2024-09-15T00:10:14.656 INFO:teuthology.orchestra.run.smithi092.stdout:5240 write 0x8834200 thru 0x88373d3 (0x31d4 bytes) 2024-09-15T00:10:14.656 INFO:teuthology.orchestra.run.smithi092.stdout:5244 write 0x64e0c00 thru 0x64e8776 (0x7b77 bytes) 2024-09-15T00:10:14.657 INFO:teuthology.orchestra.run.smithi092.stdout:5246 trunc from 0xd810000 to 0xc702a00 2024-09-15T00:10:14.690 INFO:teuthology.orchestra.run.smithi092.stdout:5248 read 0xb099000 thru 0xb0a20b7 (0x90b8 bytes) 2024-09-15T00:10:14.690 INFO:teuthology.orchestra.run.smithi092.stdout:5249 punch from 0x5461600 to 0x5470a00, (0xf400 bytes) 2024-09-15T00:10:14.690 INFO:teuthology.orchestra.run.smithi092.stdout:5251 read 0x70e6800 thru 0x70e6927 (0x128 bytes) 2024-09-15T00:10:14.690 INFO:teuthology.orchestra.run.smithi092.stdout:5256 punch from 0x8106a00 to 0x8113200, (0xc800 bytes) 2024-09-15T00:10:14.690 INFO:teuthology.orchestra.run.smithi092.stdout:5257 write 0x5e6ce00 thru 0x5e732e2 (0x64e3 bytes) 2024-09-15T00:10:14.694 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:14.692+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:5258 trunc from 0xc702a00 to 0x4e05000 2024-09-15T00:10:14.746 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:14.825 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:14.825 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:14.825 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:14.826 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:14.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5261 write 0x8367e00 thru 0x83736b9 (0xb8ba bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5263 write 0xd13b600 thru 0xd147b6e (0xc56f bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5264 punch from 0x89e3800 to 0x89e9400, (0x5c00 bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5265 read 0x2841800 thru 0x2844b7d (0x337e bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5266 write 0xa566600 thru 0xa56dcc6 (0x76c7 bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5268 punch from 0xbcdc800 to 0xbce8a00, (0xc200 bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5269 write 0x1ba3800 thru 0x1ba918b (0x598c bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5271 read 0x7204600 thru 0x720e59d (0x9f9e bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5273 read 0x4375000 thru 0x4378b6b (0x3b6c bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5276 punch from 0x249fc00 to 0x24ab000, (0xb400 bytes) 2024-09-15T00:10:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:5278 read 0x3379000 thru 0x337f670 (0x6671 bytes) 2024-09-15T00:10:14.830 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:14.825+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:14.927 INFO:teuthology.orchestra.run.smithi092.stdout:5279 punch from 0x8b1d600 to 0x8b28c00, (0xb600 bytes) 2024-09-15T00:10:14.927 INFO:teuthology.orchestra.run.smithi092.stdout:5280 read 0x2f91e00 thru 0x2f955a3 (0x37a4 bytes) 2024-09-15T00:10:14.929 INFO:teuthology.orchestra.run.smithi092.stdout:5282 read 0xa276000 thru 0xa27667c (0x67d bytes) 2024-09-15T00:10:14.929 INFO:teuthology.orchestra.run.smithi092.stdout:5284 write 0x6c3c600 thru 0x6c49322 (0xcd23 bytes) 2024-09-15T00:10:14.929 INFO:teuthology.orchestra.run.smithi092.stdout:5286 write 0xc32c200 thru 0xc333fa2 (0x7da3 bytes) 2024-09-15T00:10:14.929 INFO:teuthology.orchestra.run.smithi092.stdout:5288 write 0x5b7a600 thru 0x5b7c4e8 (0x1ee9 bytes) 2024-09-15T00:10:14.930 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:14.927+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:14.932 INFO:teuthology.orchestra.run.smithi092.stdout:5292 trunc from 0xd147c00 to 0xa9c6600 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956924209999997, 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:10:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:10:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:10:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956903955000001, 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956884209000002, 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956864259, 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956836758999998, 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956819918000001, 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956803919999999, 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956780932000001, 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956755047999998, 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956738621, 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956724350000002, 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.038 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.956715033000002, 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.039 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:15.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.955046021000001, 2024-09-15T00:10:15.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:15.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:15.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.955010958000003, 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.048 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:15.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.052 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.954985931000003, 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:15.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.954961623000003, 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:15.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.061 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2674 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3768320~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.952815276999999, 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059850591000000002, 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2674 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031670+0000", 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031671+0000", 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031682+0000", 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1248e-05 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031693+0000", 2024-09-15T00:10:15.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1395e-05 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031705+0000", 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2106000000000001e-05 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091352+0000", 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059646421999999998 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091441+0000", 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9174999999999999e-05 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091521+0000", 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9636999999999997e-05 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.066 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.952769371999999, 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:10:15.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:10:15.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.952745239999999, 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.952715867999999, 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.076 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:15.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:15.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:15.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.080 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.080 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:15.081 INFO:teuthology.orchestra.run.smithi092.stdout:5297 write 0x8aa5a00 thru 0x8aad6ab (0x7cac bytes) 2024-09-15T00:10:15.082 INFO:teuthology.orchestra.run.smithi092.stdout:5298 trunc from 0xa9c6600 to 0x9535000 2024-09-15T00:10:15.082 INFO:teuthology.orchestra.run.smithi092.stdout:5299 trunc from 0x9535000 to 0x662200 2024-09-15T00:10:15.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:15.083 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:15.079+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:15.102 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:10:15.182 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:15.180+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:15.207 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-15T00:10:15.207 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:10:15.207 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T00:10:15.208 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 1.000000 2024-09-15T00:10:15.208 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-15T00:10:15.213 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:15.214 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:15.214 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:15.214 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:15.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:5300 write 0xc533800 thru 0xc5417a6 (0xdfa7 bytes) 2024-09-15T00:10:15.233 INFO:teuthology.orchestra.run.smithi092.stdout:5303 punch from 0x1eb200 to 0x1f3200, (0x8000 bytes) 2024-09-15T00:10:15.243 INFO:teuthology.orchestra.run.smithi092.stdout:5304 trunc from 0xc541800 to 0x8991800 2024-09-15T00:10:15.283 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:15.281+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:15.375 INFO:teuthology.orchestra.run.smithi092.stdout:5305 write 0x25eec00 thru 0x25fe39f (0xf7a0 bytes) 2024-09-15T00:10:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:5306 read 0x50b5200 thru 0x50c2cd0 (0xdad1 bytes) 2024-09-15T00:10:15.376 INFO:teuthology.orchestra.run.smithi092.stdout:5307 read 0x7e2ce00 thru 0x7e313cb (0x45cc bytes) 2024-09-15T00:10:15.383 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:15.382+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:15.398 INFO:teuthology.orchestra.run.smithi092.stdout:5308 write 0x9505800 thru 0x950c7c1 (0x6fc2 bytes) 2024-09-15T00:10:15.408 INFO:teuthology.orchestra.run.smithi092.stdout:5310 trunc from 0x950c800 to 0xfee200 2024-09-15T00:10:15.409 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:15.409 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:15.409 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:15.409 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:15.409 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:15.422 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:15.461 INFO:teuthology.orchestra.run.smithi092.stdout:5311 write 0xe526a00 thru 0xe5270a6 (0x6a7 bytes) 2024-09-15T00:10:15.466 INFO:teuthology.orchestra.run.smithi092.stdout:5314 clone 35 order 23 su 262144 sc 2 2024-09-15T00:10:15.484 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:15.483+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:15.534 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.506319242000004, 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.506297324000002, 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.506276051999997, 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.504534792000001, 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.504512886000001, 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.504490793999999, 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.504459054999998, 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.504160851999998, 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:15.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:15.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.504127146999998, 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:15.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:15.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:15.578 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.499825074, 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.499500204, 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:15.587 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.499089798999997, 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.588 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:15.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.499034895999998, 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.597 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.499002861000001, 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:15.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.497525031000002, 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:15.602 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:15.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.497503316, 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.607 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:15.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:15.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:15.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:15.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.497481837000002, 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:15.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.495468733000003, 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:15.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:15.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.495449092000001, 2024-09-15T00:10:15.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:15.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:15.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.495402544000001, 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:15.626 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:15.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:15.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:15.630 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:15.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:15.633 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:15.630+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:15.733 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:15.731+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:15.739 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:15.739 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:15.740 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:15.740 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:15.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:15.768 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:15.833 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:15.832+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:15.875 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:15.891 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:15.891 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:15.891 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:15.891 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:15.891 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:15.892 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:15.892 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:15.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:15.909 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:15.909 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:15.941 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:15.939+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:16.035 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:16.034+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:16.044 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:16.044 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.130263877, 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:16.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.049 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1302435710000001, 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:16.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1302216309999999, 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.054 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:16.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:16.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1283638599999999, 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:16.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.128334181, 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:16.063 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.128311088, 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:16.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.072 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.128285915, 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:16.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:16.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1282652740000001, 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:16.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.081 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1282447330000001, 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:16.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.128221251, 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:16.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5980 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4403200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.090 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916044+0000", 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1263688649999999, 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042430530000000001, 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5980 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916044+0000", 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916044+0000", 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916044+0000", 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.11e-07 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916045+0000", 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.04e-06 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916050+0000", 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2080000000000002e-06 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916052+0000", 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.695e-06 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958378+0000", 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042325944999999997 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958426+0000", 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7783e-05 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958475+0000", 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8647999999999999e-05 2024-09-15T00:10:16.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.126357595, 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:16.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:16.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.126327826, 2024-09-15T00:10:16.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:16.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.103 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1263056469999999, 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.104 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1262783970000001, 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:16.108 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:16.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:16.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1262518029999999, 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.113 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:16.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1262324699999999, 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5997 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5099520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918218+0000", 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.124195133, 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042402333, 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5997 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918218+0000", 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918218+0000", 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918218+0000", 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:16.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918219+0000", 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.11e-06 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918223+0000", 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9299999999999996e-06 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918226+0000", 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6299999999999998e-06 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960527+0000", 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042301331999999997 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960570+0000", 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2579e-05 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960620+0000", 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0439999999999998e-05 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.124184061, 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.126 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:16.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.130 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.124168828, 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:16.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:16.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:16.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:16.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:16.135 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:16.137 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-15T00:10:16.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:16.138 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:16.138 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:16.138 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:16.134+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:16.147 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:16.147 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:16.165 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:16.165 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:16.168 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:16.168 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:16.193 INFO:teuthology.orchestra.run.smithi092.stdout:leaving image image_client.0-clone34 intact 2024-09-15T00:10:16.237 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:16.235+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:16.245 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:16.245 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:16.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:16.338 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:16.337+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:16.403 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:16.403 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:16.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:16.453 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:16.451+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:16.540 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:16.538+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:16.559 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:16.560 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:16.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:16.584 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-15T00:10:16.641 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:16.639+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:16.643 INFO:teuthology.orchestra.run.smithi092.stdout:5318 read 0xb9df400 thru 0xb9e3903 (0x4504 bytes) 2024-09-15T00:10:16.643 INFO:teuthology.orchestra.run.smithi092.stdout:5320 read 0xe05f000 thru 0xe06acc7 (0xbcc8 bytes) 2024-09-15T00:10:16.644 INFO:teuthology.orchestra.run.smithi092.stdout:5322 clone 36 order 25 su 262144 sc 13 2024-09-15T00:10:16.673 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:16.673 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:16.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:16.742 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:16.740+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:16.784 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:16.784 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:16.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:16.876 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:16.875+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:16.936 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:16.936 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:16.942 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:16.940+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:16.945 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:17.043 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:17.041+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:17.062 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:17.062 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:17.071 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:17.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:10:17.154 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:17.152+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:17.200 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone35 from 0xe527200 (overlap 0xe527200) to 0x2dd4a00 2024-09-15T00:10:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:17.223 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:17.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:17.246 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:17.244+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:17.346 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:17.347 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:17.345+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:17.355 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:17.447 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:17.445+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:17.462 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:17.462 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:17.471 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:10:17.470+0000 7f98927fc640 0 --2- 172.21.15.27:0/2353121150 >> [v2:172.21.15.27:3301/0,v1:172.21.15.27:6790/0] conn(0x7f989409bdd0 0x7f98941cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T00:10:17.548 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:17.546+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:17.581 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:17.581 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:17.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:17.597 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-15T00:10:17.649 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:17.648+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:17.750 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:17.748+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:17.760 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:17.761 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:17.771 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:17.851 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:17.849+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:17.923 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:17.924 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:17.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:17.952 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:17.950+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:18.052 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:18.051+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:10:18.127 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:18.127 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:18.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:18.169 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:18.181 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:18.179+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:18.254 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:18.252+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:18.308 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:18.316 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:18.317 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:18.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:18.329 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:18.358 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:18.359 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:18.370 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:18.368+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:18.380 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:18.380 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:18.397 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:18.415 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:18.419 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:18.419 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:18.457 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:18.455+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:18.475 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:18.475 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:18.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:18.557 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:18.556+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:18.587 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:18.588 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:18.588 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:18.588 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:18.588 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:18.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:18.597 INFO:teuthology.orchestra.run.smithi092.stdout:5323 write 0xde96200 thru 0xde97406 (0x1207 bytes) 2024-09-15T00:10:18.597 INFO:teuthology.orchestra.run.smithi092.stdout:5325 trunc from 0xe527200 to 0xe177600 2024-09-15T00:10:18.681 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:18.679+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1772 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5373952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.383138+0000", 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3617393839999998, 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044949414, 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1772 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383138+0000", 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383138+0000", 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383138+0000", 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383140+0000", 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.951e-06 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383147+0000", 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4999999999999996e-06 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383152+0000", 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8930000000000004e-06 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.427906+0000", 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044753715999999999 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.427993+0000", 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7021000000000006e-05 2024-09-15T00:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.428087+0000", 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4938000000000005e-05 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1773 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5414912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.383158+0000", 2024-09-15T00:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3617195100000004, 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045162009000000003, 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1773 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383158+0000", 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383158+0000", 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.384928+0000", 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001769863 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.384932+0000", 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7230000000000002e-06 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.384941+0000", 2024-09-15T00:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6609999999999995e-06 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.384947+0000", 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.268e-06 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.428132+0000", 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043184563000000002 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.428216+0000", 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4244000000000002e-05 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.428320+0000", 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103687 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1781 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5742592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385112+0000", 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.359765779, 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044888666000000001, 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1781 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385112+0000", 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385112+0000", 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385112+0000", 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385114+0000", 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.84e-06 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385120+0000", 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9070000000000004e-06 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385124+0000", 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5770000000000004e-06 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.429881+0000", 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044756706 2024-09-15T00:10:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.429945+0000", 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4177000000000006e-05 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430000+0000", 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5021000000000001e-05 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1782 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5783552~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385130+0000", 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3597477180000004, 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045117168999999999, 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1782 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385130+0000", 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385130+0000", 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385130+0000", 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.65e-07 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385132+0000", 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7549999999999999e-06 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385138+0000", 2024-09-15T00:10:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6699999999999999e-06 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385142+0000", 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6299999999999997e-06 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430043+0000", 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044900936000000002 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430143+0000", 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010037799999999999 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430247+0000", 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103435 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1783 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5824512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385147+0000", 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3597302139999998, 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045261416999999998, 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1783 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385147+0000", 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385147+0000", 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385148+0000", 2024-09-15T00:10:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385149+0000", 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6780000000000001e-06 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385155+0000", 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385159+0000", 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4449999999999998e-06 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430262+0000", 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045102237000000003 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430326+0000", 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4715999999999995e-05 2024-09-15T00:10:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430409+0000", 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2415000000000003e-05 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1784 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5865472~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385164+0000", 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3597131019999997, 2024-09-15T00:10:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045458331999999997, 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1784 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385164+0000", 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385164+0000", 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385165+0000", 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3799999999999998e-07 2024-09-15T00:10:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385166+0000", 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.685e-06 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385172+0000", 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5609999999999998e-06 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385176+0000", 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430453+0000", 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045276755000000002 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430540+0000", 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430623+0000", 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2479999999999996e-05 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1785 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5906432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385181+0000", 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3596960439999997, 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045656909000000002, 2024-09-15T00:10:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1785 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385181+0000", 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385181+0000", 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385182+0000", 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385183+0000", 2024-09-15T00:10:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.748e-06 2024-09-15T00:10:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385189+0000", 2024-09-15T00:10:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1279999999999999e-06 2024-09-15T00:10:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385193+0000", 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.493e-06 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430669+0000", 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045475504 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430752+0000", 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3548999999999996e-05 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430838+0000", 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6143999999999994e-05 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1786 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5947392~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385206+0000", 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3596709999999996, 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045836012000000002, 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1786 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385206+0000", 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385206+0000", 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385207+0000", 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385209+0000", 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9769999999999999e-06 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385215+0000", 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6110000000000001e-06 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385219+0000", 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4290000000000003e-06 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430875+0000", 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045655880000000003 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430954+0000", 2024-09-15T00:10:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9406000000000005e-05 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431042+0000", 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8170000000000005e-05 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1787 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5988352~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385225+0000", 2024-09-15T00:10:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3596522540000002, 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045998764999999997, 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1787 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385225+0000", 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385225+0000", 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385225+0000", 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0899999999999997e-07 2024-09-15T00:10:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385227+0000", 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.962e-06 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385235+0000", 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6669999999999996e-06 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385240+0000", 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.95e-06 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431073+0000", 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045832497999999999 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431144+0000", 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1464999999999994e-05 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431224+0000", 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9913999999999994e-05 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1788 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 6029312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385247+0000", 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3596308629999996, 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046194975999999999, 2024-09-15T00:10:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1788 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385247+0000", 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385247+0000", 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385247+0000", 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385249+0000", 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.995e-06 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385255+0000", 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385260+0000", 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431266+0000", 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046005742000000002 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431349+0000", 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2737000000000004e-05 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431442+0000", 2024-09-15T00:10:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2927e-05 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1789 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 6070272~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385266+0000", 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3596113829999998, 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046303922999999997, 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1789 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385266+0000", 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385266+0000", 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.386929+0000", 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016625030000000001 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.386933+0000", 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3880000000000002e-06 2024-09-15T00:10:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.386945+0000", 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620999999999999e-05 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.386952+0000", 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5800000000000003e-06 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431477+0000", 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044525153999999997 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431519+0000", 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347000000000002e-05 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431570+0000", 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1329999999999998e-05 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1792 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 6193152~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387027+0000", 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3578509040000002, 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045050564000000001, 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1792 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387027+0000", 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387027+0000", 2024-09-15T00:10:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387027+0000", 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-07 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387029+0000", 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387037+0000", 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4309999999999997e-06 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387049+0000", 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2468e-05 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431901+0000", 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044851374999999999 2024-09-15T00:10:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431991+0000", 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0056999999999995e-05 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432077+0000", 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6379999999999996e-05 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1793 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 6234112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387058+0000", 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3578195150000001, 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045238681000000003, 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1793 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387058+0000", 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387058+0000", 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387058+0000", 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387061+0000", 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7379999999999999e-06 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387069+0000", 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387075+0000", 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3499999999999996e-06 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432110+0000", 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045035155 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432188+0000", 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8187000000000002e-05 2024-09-15T00:10:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432297+0000", 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108559 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1794 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387097+0000", 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3577803849999999, 2024-09-15T00:10:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241772999999999, 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1794 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387097+0000", 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387097+0000", 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387098+0000", 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387100+0000", 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.39e-06 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387107+0000", 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1600000000000001e-06 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387113+0000", 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4829999999999999e-06 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432330+0000", 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045217368000000001 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432339+0000", 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8310000000000007e-06 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1795 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387120+0000", 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3577578969999999, 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045243522000000001, 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1795 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387120+0000", 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387120+0000", 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387120+0000", 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387122+0000", 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0779999999999998e-06 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387128+0000", 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0680000000000002e-06 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387133+0000", 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.797e-06 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432359+0000", 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045225564000000003 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432363+0000", 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5480000000000003e-06 2024-09-15T00:10:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1796 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387140+0000", 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3577374019999997, 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045255984999999999, 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1796 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387140+0000", 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387140+0000", 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387140+0000", 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387143+0000", 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387149+0000", 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7259999999999996e-06 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387154+0000", 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0350000000000001e-06 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432388+0000", 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045234116999999997 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432396+0000", 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1797 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387161+0000", 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3577162810000001, 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045252725000000001, 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1797 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387161+0000", 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387161+0000", 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387162+0000", 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387164+0000", 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-15T00:10:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387170+0000", 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3049999999999998e-06 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387175+0000", 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8489999999999997e-06 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432408+0000", 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045233425000000001 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432414+0000", 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7119999999999997e-06 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1798 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387180+0000", 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.357697463, 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045265596999999998, 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1798 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387180+0000", 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387180+0000", 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387180+0000", 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387182+0000", 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.804e-06 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387188+0000", 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6919999999999999e-06 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387192+0000", 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.639e-06 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432437+0000", 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045244541999999999 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432446+0000", 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5599999999999994e-06 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1799 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387198+0000", 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3576790580000004, 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045270091999999998, 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1799 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387198+0000", 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387198+0000", 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387199+0000", 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7500000000000001e-07 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387201+0000", 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.928e-06 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387206+0000", 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:10:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387211+0000", 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7600000000000002e-06 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432464+0000", 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045253267999999999 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432468+0000", 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1200000000000004e-06 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1800 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387216+0000", 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3576612050000003, 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045283179, 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1800 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387216+0000", 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387216+0000", 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.388928+0000", 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001712019 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.388933+0000", 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2910000000000001e-06 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.388944+0000", 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1834000000000001e-05 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.388955+0000", 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205e-05 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432492+0000", 2024-09-15T00:10:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043537449999999998 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432499+0000", 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3799999999999996e-06 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:18.843 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:18.845 INFO:teuthology.orchestra.run.smithi092.stdout:5329 read 0x86db800 thru 0x86dc9c4 (0x11c5 bytes) 2024-09-15T00:10:18.845 INFO:teuthology.orchestra.run.smithi092.stdout:5331 trunc from 0xe177600 to 0x4b89e00 2024-09-15T00:10:18.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:18.846 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:18.843+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:18.867 INFO:teuthology.orchestra.run.smithi092.stdout:5333 read 0x43fc200 thru 0x4405920 (0x9721 bytes) 2024-09-15T00:10:18.868 INFO:teuthology.orchestra.run.smithi092.stdout:5336 read 0x1545200 thru 0x154ddd6 (0x8bd7 bytes) 2024-09-15T00:10:18.869 INFO:teuthology.orchestra.run.smithi092.stdout:5338 read 0x863c00 thru 0x86ef48 (0xb349 bytes) 2024-09-15T00:10:18.872 INFO:teuthology.orchestra.run.smithi092.stdout:5340 punch from 0x1db2000 to 0x1dbce00, (0xae00 bytes) 2024-09-15T00:10:18.872 INFO:teuthology.orchestra.run.smithi092.stdout:5342 punch from 0x47eca00 to 0x47f5800, (0x8e00 bytes) 2024-09-15T00:10:18.883 INFO:teuthology.orchestra.run.smithi092.stdout:5343 write 0x8b90200 thru 0x8b974b2 (0x72b3 bytes) 2024-09-15T00:10:18.900 INFO:teuthology.orchestra.run.smithi092.stdout:5346 write 0xba84200 thru 0xba86cef (0x2af0 bytes) 2024-09-15T00:10:18.904 INFO:teuthology.orchestra.run.smithi092.stdout:5347 trunc from 0xba86e00 to 0x8d60600 2024-09-15T00:10:18.931 INFO:teuthology.orchestra.run.smithi092.stdout:5351 write 0xb4d2000 thru 0xb4def5e (0xcf5f bytes) 2024-09-15T00:10:18.932 INFO:teuthology.orchestra.run.smithi092.stdout:5354 punch from 0x2e6d600 to 0x2e75a00, (0x8400 bytes) 2024-09-15T00:10:18.933 INFO:teuthology.orchestra.run.smithi092.stdout:5358 write 0x395f000 thru 0x3967955 (0x8956 bytes) 2024-09-15T00:10:18.934 INFO:teuthology.orchestra.run.smithi092.stdout:5360 write 0x337da00 thru 0x33866c9 (0x8cca bytes) 2024-09-15T00:10:18.941 INFO:teuthology.orchestra.run.smithi092.stdout:5361 trunc from 0xb4df000 to 0x5849c00 2024-09-15T00:10:18.945 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:18.944+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:18.957 INFO:teuthology.orchestra.run.smithi092.stdout:5365 read 0xed9a00 thru 0xee6ade (0xd0df bytes) 2024-09-15T00:10:18.973 INFO:teuthology.orchestra.run.smithi092.stdout:5366 write 0xb703800 thru 0xb710acb (0xd2cc bytes) 2024-09-15T00:10:18.974 INFO:teuthology.orchestra.run.smithi092.stdout:5367 write 0x273a800 thru 0x273c253 (0x1a54 bytes) 2024-09-15T00:10:18.976 INFO:teuthology.orchestra.run.smithi092.stdout:5368 read 0x4e36800 thru 0x4e414cc (0xaccd bytes) 2024-09-15T00:10:18.977 INFO:teuthology.orchestra.run.smithi092.stdout:5369 write 0x1853400 thru 0x1856d1a (0x391b bytes) 2024-09-15T00:10:18.977 INFO:teuthology.orchestra.run.smithi092.stdout:5371 punch from 0x8851400 to 0x8852e00, (0x1a00 bytes) 2024-09-15T00:10:18.977 INFO:teuthology.orchestra.run.smithi092.stdout:5374 read 0x8754200 thru 0x875e413 (0xa214 bytes) 2024-09-15T00:10:18.978 INFO:teuthology.orchestra.run.smithi092.stdout:5377 read 0x5138400 thru 0x5147050 (0xec51 bytes) 2024-09-15T00:10:18.980 INFO:teuthology.orchestra.run.smithi092.stdout:5378 read 0xa721200 thru 0xa72cabe (0xb8bf bytes) 2024-09-15T00:10:18.980 INFO:teuthology.orchestra.run.smithi092.stdout:5379 write 0x715a400 thru 0x716772c (0xd32d bytes) 2024-09-15T00:10:18.981 INFO:teuthology.orchestra.run.smithi092.stdout:5381 read 0xb0d4600 thru 0xb0d7710 (0x3111 bytes) 2024-09-15T00:10:18.983 INFO:teuthology.orchestra.run.smithi092.stdout:5382 trunc from 0xb710c00 to 0xce5ec00 2024-09-15T00:10:18.991 INFO:teuthology.orchestra.run.smithi092.stdout:5385 write 0x23d5600 thru 0x23e3855 (0xe256 bytes) 2024-09-15T00:10:18.997 INFO:teuthology.orchestra.run.smithi092.stdout:5388 write 0x99aec00 thru 0x99b7c87 (0x9088 bytes) 2024-09-15T00:10:18.998 INFO:teuthology.orchestra.run.smithi092.stdout:5392 punch from 0x2253600 to 0x2258e00, (0x5800 bytes) 2024-09-15T00:10:18.998 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:18.998 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:18.998 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:18.998 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:19.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:19.010 INFO:teuthology.orchestra.run.smithi092.stdout:5393 trunc from 0xce5ec00 to 0x2df8800 2024-09-15T00:10:19.041 INFO:teuthology.orchestra.run.smithi092.stdout:5395 trunc from 0x2df8800 to 0x4548400 2024-09-15T00:10:19.046 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:19.044+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:19.056 INFO:teuthology.orchestra.run.smithi092.stdout:5399 punch from 0xe4000 to 0xf2c00, (0xec00 bytes) 2024-09-15T00:10:19.081 INFO:teuthology.orchestra.run.smithi092.stdout:5400 write 0xceb2c00 thru 0xcec1fcb (0xf3cc bytes) 2024-09-15T00:10:19.083 INFO:teuthology.orchestra.run.smithi092.stdout:5403 write 0x377a800 thru 0x3782357 (0x7b58 bytes) 2024-09-15T00:10:19.083 INFO:teuthology.orchestra.run.smithi092.stdout:5410 read 0x7c36a00 thru 0x7c41fb1 (0xb5b2 bytes) 2024-09-15T00:10:19.086 INFO:teuthology.orchestra.run.smithi092.stdout:5413 write 0x2033600 thru 0x203f1d5 (0xbbd6 bytes) 2024-09-15T00:10:19.091 INFO:teuthology.orchestra.run.smithi092.stdout:5415 trunc from 0xcec2000 to 0x69d4c00 2024-09-15T00:10:19.109 INFO:teuthology.orchestra.run.smithi092.stdout:5417 read 0x2f27c00 thru 0x2f33edc (0xc2dd bytes) 2024-09-15T00:10:19.114 INFO:teuthology.orchestra.run.smithi092.stdout:5419 punch from 0x5132c00 to 0x513e800, (0xbc00 bytes) 2024-09-15T00:10:19.128 INFO:teuthology.orchestra.run.smithi092.stdout:5421 write 0x9933400 thru 0x9937b3a (0x473b bytes) 2024-09-15T00:10:19.130 INFO:teuthology.orchestra.run.smithi092.stdout:5426 write 0x5574200 thru 0x55783ed (0x41ee bytes) 2024-09-15T00:10:19.130 INFO:teuthology.orchestra.run.smithi092.stdout:5427 read 0x563c600 thru 0x563e80e (0x220f bytes) 2024-09-15T00:10:19.133 INFO:teuthology.orchestra.run.smithi092.stdout:5428 read 0x7abc600 thru 0x7ac6bfd (0xa5fe bytes) 2024-09-15T00:10:19.134 INFO:teuthology.orchestra.run.smithi092.stdout:5429 write 0x6487600 thru 0x648eabb (0x74bc bytes) 2024-09-15T00:10:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:5430 read 0x45ed200 thru 0x45f370d (0x650e bytes) 2024-09-15T00:10:19.135 INFO:teuthology.orchestra.run.smithi092.stdout:5431 punch from 0x5a96a00 to 0x5a9b400, (0x4a00 bytes) 2024-09-15T00:10:19.146 INFO:teuthology.orchestra.run.smithi092.stdout:5434 write 0x9db7200 thru 0x9dc00dd (0x8ede bytes) 2024-09-15T00:10:19.147 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:19.145+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:19.147 INFO:teuthology.orchestra.run.smithi092.stdout:5436 read 0x142f200 thru 0x143ee02 (0xfc03 bytes) 2024-09-15T00:10:19.148 INFO:teuthology.orchestra.run.smithi092.stdout:5437 punch from 0x9561e00 to 0x9562800, (0xa00 bytes) 2024-09-15T00:10:19.151 INFO:teuthology.orchestra.run.smithi092.stdout:5439 write 0x3acbc00 thru 0x3ada8e4 (0xece5 bytes) 2024-09-15T00:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:19.161 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:19.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:19.171 INFO:teuthology.orchestra.run.smithi092.stdout:5441 write 0xc282400 thru 0xc289de3 (0x79e4 bytes) 2024-09-15T00:10:19.171 INFO:teuthology.orchestra.run.smithi092.stdout:5442 punch from 0x4184000 to 0x418b400, (0x7400 bytes) 2024-09-15T00:10:19.171 INFO:teuthology.orchestra.run.smithi092.stdout:5443 read 0xc134c00 thru 0xc134f6f (0x370 bytes) 2024-09-15T00:10:19.172 INFO:teuthology.orchestra.run.smithi092.stdout:5447 punch from 0x4261e00 to 0x4271a00, (0xfc00 bytes) 2024-09-15T00:10:19.189 INFO:teuthology.orchestra.run.smithi092.stdout:5450 write 0xd9ba000 thru 0xd9c07e4 (0x67e5 bytes) 2024-09-15T00:10:19.189 INFO:teuthology.orchestra.run.smithi092.stdout:5451 punch from 0xd14b600 to 0xd158e00, (0xd800 bytes) 2024-09-15T00:10:19.189 INFO:teuthology.orchestra.run.smithi092.stdout:5454 read 0xa8cfc00 thru 0xa8d089a (0xc9b bytes) 2024-09-15T00:10:19.192 INFO:teuthology.orchestra.run.smithi092.stdout:5455 punch from 0x542d600 to 0x543c400, (0xee00 bytes) 2024-09-15T00:10:19.192 INFO:teuthology.orchestra.run.smithi092.stdout:5458 read 0x3223c00 thru 0x322bd39 (0x813a bytes) 2024-09-15T00:10:19.210 INFO:teuthology.orchestra.run.smithi092.stdout:5459 write 0xdcd4c00 thru 0xdcdea91 (0x9e92 bytes) 2024-09-15T00:10:19.223 INFO:teuthology.orchestra.run.smithi092.stdout:5462 trunc from 0xdcdec00 to 0x2395400 2024-09-15T00:10:19.248 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:19.246+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:19.254 INFO:teuthology.orchestra.run.smithi092.stdout:5463 read 0x12c9e00 thru 0x12d3087 (0x9288 bytes) 2024-09-15T00:10:19.255 INFO:teuthology.orchestra.run.smithi092.stdout:5465 read 0x1590800 thru 0x159a446 (0x9c47 bytes) 2024-09-15T00:10:19.269 INFO:teuthology.orchestra.run.smithi092.stdout:5466 trunc from 0x2395400 to 0xd859a00 2024-09-15T00:10:19.279 INFO:teuthology.orchestra.run.smithi092.stdout:5468 read 0x8554000 thru 0x855a0aa (0x60ab bytes) 2024-09-15T00:10:19.281 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:19.281 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:19.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:19.281 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:19.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.281 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.396294296999997, 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:19.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:19.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.396255666999998, 2024-09-15T00:10:19.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:19.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.290 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.396229730000002, 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.291 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:19.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:19.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.396213662000001, 2024-09-15T00:10:19.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:19.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:19.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.396072566999997, 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.300 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:19.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.395215780999997, 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:19.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.306 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:19.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:19.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.395180764999999, 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:19.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.395155162000002, 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:19.314 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:19.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.394980896, 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.319 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:19.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:19.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:19.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.394749757, 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:19.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:19.328 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.393937813000001, 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:19.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.393858072999997, 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:19.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.393653047999997, 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:19.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:19.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:19.342 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.393441514999999, 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:19.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.390885507999997, 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:19.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.388902401999999, 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.352 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:19.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.388882830999997, 2024-09-15T00:10:19.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:19.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:19.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.118139980999999, 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:19.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.212388486, 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:19.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:19.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:19.370 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.595301046, 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:19.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:19.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:19.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:19.376 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:19.376 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:19.378 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:19.375+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:19.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:19.379 INFO:teuthology.orchestra.run.smithi092.stdout:5470 read 0xca0b200 thru 0xca198d5 (0xe6d6 bytes) 2024-09-15T00:10:19.379 INFO:teuthology.orchestra.run.smithi092.stdout:5472 read 0x6c5b600 thru 0x6c6af48 (0xf949 bytes) 2024-09-15T00:10:19.379 INFO:teuthology.orchestra.run.smithi092.stdout:5474 punch from 0x4595000 to 0x459c800, (0x7800 bytes) 2024-09-15T00:10:19.379 INFO:teuthology.orchestra.run.smithi092.stdout:5475 read 0xce89200 thru 0xce8feea (0x6ceb bytes) 2024-09-15T00:10:19.379 INFO:teuthology.orchestra.run.smithi092.stdout:5476 punch from 0x6458e00 to 0x6468c00, (0xfe00 bytes) 2024-09-15T00:10:19.379 INFO:teuthology.orchestra.run.smithi092.stdout:5480 trunc from 0xd859a00 to 0x55b1000 2024-09-15T00:10:19.380 INFO:teuthology.orchestra.run.smithi092.stdout:5481 write 0x1ba8000 thru 0x1ba83f4 (0x3f5 bytes) 2024-09-15T00:10:19.380 INFO:teuthology.orchestra.run.smithi092.stdout:5486 read 0x2923000 thru 0x2927c2c (0x4c2d bytes) 2024-09-15T00:10:19.380 INFO:teuthology.orchestra.run.smithi092.stdout:5487 trunc from 0x55b1000 to 0x1efa400 2024-09-15T00:10:19.380 INFO:teuthology.orchestra.run.smithi092.stdout:5488 read 0x1985a00 thru 0x1987ab6 (0x20b7 bytes) 2024-09-15T00:10:19.380 INFO:teuthology.orchestra.run.smithi092.stdout:5490 read 0x10afc00 thru 0x10b8301 (0x8702 bytes) 2024-09-15T00:10:19.380 INFO:teuthology.orchestra.run.smithi092.stdout:5492 read 0x1c77000 thru 0x1c85bc1 (0xebc2 bytes) 2024-09-15T00:10:19.478 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:19.477+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:19.487 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:19.487 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:19.487 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:19.487 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:19.495 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:19.556 INFO:teuthology.orchestra.run.smithi092.stdout:5497 write 0x97c7c00 thru 0x97d5521 (0xd922 bytes) 2024-09-15T00:10:19.556 INFO:teuthology.orchestra.run.smithi092.stdout:5500 punch from 0x2268200 to 0x2274200, (0xc000 bytes) 2024-09-15T00:10:19.564 INFO:teuthology.orchestra.run.smithi092.stdout:5501 trunc from 0x97d5600 to 0x2631400 2024-09-15T00:10:19.578 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:19.577+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:19.585 INFO:teuthology.orchestra.run.smithi092.stdout:5503 read 0x1a72400 thru 0x1a7a8a1 (0x84a2 bytes) 2024-09-15T00:10:19.593 INFO:teuthology.orchestra.run.smithi092.stdout:5504 read 0x216d200 thru 0x2177773 (0xa574 bytes) 2024-09-15T00:10:19.594 INFO:teuthology.orchestra.run.smithi092.stdout:5505 read 0x68400 thru 0x69b1c (0x171d bytes) 2024-09-15T00:10:19.594 INFO:teuthology.orchestra.run.smithi092.stdout:5507 write 0xbae00 thru 0xbf352 (0x4553 bytes) 2024-09-15T00:10:19.605 INFO:teuthology.orchestra.run.smithi092.stdout:5509 trunc from 0x2631400 to 0xb2a0800 2024-09-15T00:10:19.606 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:19.606 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:19.607 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:19.617 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:19.621 INFO:teuthology.orchestra.run.smithi092.stdout:5511 punch from 0x8550400 to 0x8553600, (0x3200 bytes) 2024-09-15T00:10:19.622 INFO:teuthology.orchestra.run.smithi092.stdout:5512 read 0x6b6a00 thru 0x6c17b9 (0xadba bytes) 2024-09-15T00:10:19.624 INFO:teuthology.orchestra.run.smithi092.stdout:5515 read 0x671f200 thru 0x67246d4 (0x54d5 bytes) 2024-09-15T00:10:19.624 INFO:teuthology.orchestra.run.smithi092.stdout:5516 write 0x7455600 thru 0x745cf9e (0x799f bytes) 2024-09-15T00:10:19.642 INFO:teuthology.orchestra.run.smithi092.stdout:5517 write 0xbfa9400 thru 0xbfb412f (0xad30 bytes) 2024-09-15T00:10:19.642 INFO:teuthology.orchestra.run.smithi092.stdout:5518 write 0x4898600 thru 0x489bd18 (0x3719 bytes) 2024-09-15T00:10:19.655 INFO:teuthology.orchestra.run.smithi092.stdout:5522 write 0xd773c00 thru 0xd778b3f (0x4f40 bytes) 2024-09-15T00:10:19.657 INFO:teuthology.orchestra.run.smithi092.stdout:5527 write 0x1af0000 thru 0x1af60f7 (0x60f8 bytes) 2024-09-15T00:10:19.657 INFO:teuthology.orchestra.run.smithi092.stdout:5528 punch from 0xb066800 to 0xb06c800, (0x6000 bytes) 2024-09-15T00:10:19.658 INFO:teuthology.orchestra.run.smithi092.stdout:5529 read 0xd1b2a00 thru 0xd1b466b (0x1c6c bytes) 2024-09-15T00:10:19.659 INFO:teuthology.orchestra.run.smithi092.stdout:5532 read 0x92ba00 thru 0x93a506 (0xeb07 bytes) 2024-09-15T00:10:19.659 INFO:teuthology.orchestra.run.smithi092.stdout:5534 read 0x202ca00 thru 0x20322c9 (0x58ca bytes) 2024-09-15T00:10:19.660 INFO:teuthology.orchestra.run.smithi092.stdout:5538 read 0xb39f200 thru 0xb3a749e (0x829f bytes) 2024-09-15T00:10:19.661 INFO:teuthology.orchestra.run.smithi092.stdout:5540 punch from 0x3d2aa00 to 0x3d35e00, (0xb400 bytes) 2024-09-15T00:10:19.662 INFO:teuthology.orchestra.run.smithi092.stdout:5541 write 0x7051c00 thru 0x70570a9 (0x54aa bytes) 2024-09-15T00:10:19.662 INFO:teuthology.orchestra.run.smithi092.stdout:5548 punch from 0x19e8c00 to 0x19f7400, (0xe800 bytes) 2024-09-15T00:10:19.662 INFO:teuthology.orchestra.run.smithi092.stdout:5549 read 0x491da00 thru 0x49264eb (0x8aec bytes) 2024-09-15T00:10:19.663 INFO:teuthology.orchestra.run.smithi092.stdout:5550 trunc from 0xd778c00 to 0xdad8800 2024-09-15T00:10:19.673 INFO:teuthology.orchestra.run.smithi092.stdout:5551 punch from 0x87af000 to 0x87af600, (0x600 bytes) 2024-09-15T00:10:19.675 INFO:teuthology.orchestra.run.smithi092.stdout:5553 write 0x13e1200 thru 0x13ea33b (0x913c bytes) 2024-09-15T00:10:19.680 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:19.678+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:19.686 INFO:teuthology.orchestra.run.smithi092.stdout:5554 trunc from 0xdad8800 to 0x52c0400 2024-09-15T00:10:19.713 INFO:teuthology.orchestra.run.smithi092.stdout:5557 read 0x1adc200 thru 0x1ae3664 (0x7465 bytes) 2024-09-15T00:10:19.714 INFO:teuthology.orchestra.run.smithi092.stdout:5558 read 0x2a96000 thru 0x2a9cc35 (0x6c36 bytes) 2024-09-15T00:10:19.714 INFO:teuthology.orchestra.run.smithi092.stdout:5560 read 0x422b800 thru 0x423207d (0x687e bytes) 2024-09-15T00:10:19.714 INFO:teuthology.orchestra.run.smithi092.stdout:5561 read 0xa7ec00 thru 0xa860d6 (0x74d7 bytes) 2024-09-15T00:10:19.714 INFO:teuthology.orchestra.run.smithi092.stdout:5563 punch from 0x473c200 to 0x4747a00, (0xb800 bytes) 2024-09-15T00:10:19.714 INFO:teuthology.orchestra.run.smithi092.stdout:5564 write 0x4c9a00 thru 0x4ce936 (0x4f37 bytes) 2024-09-15T00:10:19.725 INFO:teuthology.orchestra.run.smithi092.stdout:5565 write 0x7475000 thru 0x7477b6e (0x2b6f bytes) 2024-09-15T00:10:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:5570 read 0x5ce5e00 thru 0x5cf18c6 (0xbac7 bytes) 2024-09-15T00:10:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:5571 punch from 0x552b000 to 0x553a200, (0xf200 bytes) 2024-09-15T00:10:19.727 INFO:teuthology.orchestra.run.smithi092.stdout:5573 read 0x638c400 thru 0x6392b32 (0x6733 bytes) 2024-09-15T00:10:19.731 INFO:teuthology.orchestra.run.smithi092.stdout:5574 trunc from 0x7477c00 to 0x4460e00 2024-09-15T00:10:19.738 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:19.738 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:19.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:19.738 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:19.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.739 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.739 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:19.740 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706609581999999, 2024-09-15T00:10:19.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:10:19.740 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.740 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:10:19.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.742 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:19.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:10:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:10:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:10:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:10:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:10:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:10:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:10:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:10:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:10:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:10:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:10:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706589327000003, 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.752 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:19.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:19.756 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706569580999997, 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:19.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706549631000001, 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:19.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706522131, 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:19.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706505290000003, 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:19.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.775 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706489292000001, 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.780 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706466304000003, 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.70644042, 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.786 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706423993000001, 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:19.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706409721999997, 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:19.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:19.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.706400404999997, 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.800 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.804 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.704731393000003, 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:19.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:19.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.704696329999997, 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:19.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:19.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.704671302999998, 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:19.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:19.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.704646994999997, 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.819 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:19.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.702454744000001, 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.824 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:10:19.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.828 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.702430612000001, 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:19.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.70240124, 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.835 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:19.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.0378978130000001, 2024-09-15T00:10:19.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:19.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:10:19.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:10:19.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:19.845 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:19.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:19.848 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:19.844+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:19.849 INFO:teuthology.orchestra.run.smithi092.stdout:5576 write 0xd21fc00 thru 0xd22433e (0x473f bytes) 2024-09-15T00:10:19.849 INFO:teuthology.orchestra.run.smithi092.stdout:5579 write 0x4dfde00 thru 0x4e02c21 (0x4e22 bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5586 write 0x1d92a00 thru 0x1d93b34 (0x1135 bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5588 read 0x2233600 thru 0x223ad5c (0x775d bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5591 read 0xa81f200 thru 0xa827b3d (0x893e bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5592 trunc from 0xd224400 to 0xa7c6800 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5593 write 0x4235200 thru 0x423b557 (0x6358 bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5594 write 0x4d38400 thru 0x4d45c35 (0xd836 bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5597 read 0x6601800 thru 0x660e9c1 (0xd1c2 bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5600 punch from 0x4f09600 to 0x4f15e00, (0xc800 bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5601 write 0xea2e600 thru 0xea385b8 (0x9fb9 bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5606 write 0x5390c00 thru 0x5399dc2 (0x91c3 bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5610 write 0x809ce00 thru 0x80ab90e (0xeb0f bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5611 write 0x29b0c00 thru 0x29b726f (0x6670 bytes) 2024-09-15T00:10:19.850 INFO:teuthology.orchestra.run.smithi092.stdout:5612 punch from 0xd437a00 to 0xd43e400, (0x6a00 bytes) 2024-09-15T00:10:19.851 INFO:teuthology.orchestra.run.smithi092.stdout:5613 read 0xe715000 thru 0xe71c223 (0x7224 bytes) 2024-09-15T00:10:19.851 INFO:teuthology.orchestra.run.smithi092.stdout:5614 trunc from 0xea38600 to 0x38a6c00 2024-09-15T00:10:19.851 INFO:teuthology.orchestra.run.smithi092.stdout:5616 trunc from 0x38a6c00 to 0x3bf3600 2024-09-15T00:10:19.864 INFO:teuthology.orchestra.run.smithi092.stdout:5617 read 0x2b56a00 thru 0x2b59b91 (0x3192 bytes) 2024-09-15T00:10:19.872 INFO:teuthology.orchestra.run.smithi092.stdout:5618 punch from 0x15afc00 to 0x15be400, (0xe800 bytes) 2024-09-15T00:10:19.873 INFO:teuthology.orchestra.run.smithi092.stdout:5619 read 0x3bb7c00 thru 0x3bc1ce4 (0xa0e5 bytes) 2024-09-15T00:10:19.881 INFO:teuthology.orchestra.run.smithi092.stdout:5622 trunc from 0x3bf3600 to 0xa1d5600 2024-09-15T00:10:19.889 INFO:teuthology.orchestra.run.smithi092.stdout:5623 read 0x7485800 thru 0x7486fef (0x17f0 bytes) 2024-09-15T00:10:19.897 INFO:teuthology.orchestra.run.smithi092.stdout:5624 read 0x6cf1600 thru 0x6cf76aa (0x60ab bytes) 2024-09-15T00:10:19.897 INFO:teuthology.orchestra.run.smithi092.stdout:5625 read 0x2a1c800 thru 0x2a28c9c (0xc49d bytes) 2024-09-15T00:10:19.899 INFO:teuthology.orchestra.run.smithi092.stdout:5626 punch from 0x374c600 to 0x3759c00, (0xd600 bytes) 2024-09-15T00:10:19.911 INFO:teuthology.orchestra.run.smithi092.stdout:5627 write 0xd0a6600 thru 0xd0b3b77 (0xd578 bytes) 2024-09-15T00:10:19.919 INFO:teuthology.orchestra.run.smithi092.stdout:5628 write 0x6e2c800 thru 0x6e39aa1 (0xd2a2 bytes) 2024-09-15T00:10:19.919 INFO:teuthology.orchestra.run.smithi092.stdout:5629 punch from 0x31ac200 to 0x31ae600, (0x2400 bytes) 2024-09-15T00:10:19.920 INFO:teuthology.orchestra.run.smithi092.stdout:5631 read 0xb643200 thru 0xb649956 (0x6757 bytes) 2024-09-15T00:10:19.920 INFO:teuthology.orchestra.run.smithi092.stdout:5632 write 0xa343400 thru 0xa35124d (0xde4e bytes) 2024-09-15T00:10:19.921 INFO:teuthology.orchestra.run.smithi092.stdout:5633 read 0x32d0400 thru 0x32e0036 (0xfc37 bytes) 2024-09-15T00:10:19.921 INFO:teuthology.orchestra.run.smithi092.stdout:5639 punch from 0x71c4e00 to 0x71c7400, (0x2600 bytes) 2024-09-15T00:10:19.921 INFO:teuthology.orchestra.run.smithi092.stdout:5643 read 0x2185a00 thru 0x218767d (0x1c7e bytes) 2024-09-15T00:10:19.922 INFO:teuthology.orchestra.run.smithi092.stdout:5646 write 0x872a800 thru 0x873229a (0x7a9b bytes) 2024-09-15T00:10:19.922 INFO:teuthology.orchestra.run.smithi092.stdout:5647 read 0xc794c00 thru 0xc79ba6a (0x6e6b bytes) 2024-09-15T00:10:19.923 INFO:teuthology.orchestra.run.smithi092.stdout:5651 read 0x57f0400 thru 0x57ff94f (0xf550 bytes) 2024-09-15T00:10:19.923 INFO:teuthology.orchestra.run.smithi092.stdout:5653 read 0x3080200 thru 0x308314f (0x2f50 bytes) 2024-09-15T00:10:19.923 INFO:teuthology.orchestra.run.smithi092.stdout:5654 punch from 0x3587000 to 0x3587c00, (0xc00 bytes) 2024-09-15T00:10:19.924 INFO:teuthology.orchestra.run.smithi092.stdout:5655 read 0xbba5400 thru 0xbbb0776 (0xb377 bytes) 2024-09-15T00:10:19.924 INFO:teuthology.orchestra.run.smithi092.stdout:5657 write 0x93d6c00 thru 0x93e410a (0xd50b bytes) 2024-09-15T00:10:19.925 INFO:teuthology.orchestra.run.smithi092.stdout:5658 read 0x5d82e00 thru 0x5d8a07f (0x7280 bytes) 2024-09-15T00:10:19.925 INFO:teuthology.orchestra.run.smithi092.stdout:5659 write 0xab1b600 thru 0xab208a9 (0x52aa bytes) 2024-09-15T00:10:19.941 INFO:teuthology.orchestra.run.smithi092.stdout:5661 trunc from 0xd0b3c00 to 0x1847800 2024-09-15T00:10:19.952 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:19.946+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:19.968 INFO:teuthology.orchestra.run.smithi092.stdout:5662 trunc from 0x1847800 to 0x27de200 2024-09-15T00:10:19.988 INFO:teuthology.orchestra.run.smithi092.stdout:5665 read 0x1840200 thru 0x184ed05 (0xeb06 bytes) 2024-09-15T00:10:19.989 INFO:teuthology.orchestra.run.smithi092.stdout:5666 read 0x235fe00 thru 0x2368f44 (0x9145 bytes) 2024-09-15T00:10:19.990 INFO:teuthology.orchestra.run.smithi092.stdout:5669 write 0x11cb400 thru 0x11d6790 (0xb391 bytes) 2024-09-15T00:10:20.007 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:20.007 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:20.007 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:20.010 INFO:teuthology.orchestra.run.smithi092.stdout:5670 write 0x5c5be00 thru 0x5c6664c (0xa84d bytes) 2024-09-15T00:10:20.011 INFO:teuthology.orchestra.run.smithi092.stdout:5673 read 0x6d5000 thru 0x6d788b (0x288c bytes) 2024-09-15T00:10:20.012 INFO:teuthology.orchestra.run.smithi092.stdout:5674 read 0x3f71200 thru 0x3f80b14 (0xf915 bytes) 2024-09-15T00:10:20.013 INFO:teuthology.orchestra.run.smithi092.stdout:5676 read 0x249b800 thru 0x24a5074 (0x9875 bytes) 2024-09-15T00:10:20.014 INFO:teuthology.orchestra.run.smithi092.stdout:5677 write 0x4a01e00 thru 0x4a04cd8 (0x2ed9 bytes) 2024-09-15T00:10:20.014 INFO:teuthology.orchestra.run.smithi092.stdout:5678 clone 37 order 25 su 8388608 sc 12 2024-09-15T00:10:20.019 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:20.079 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:20.077+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:20.137 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:20.138 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:20.138 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:20.138 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:10:20.147 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:20.190 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:20.187+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:20.221 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone36 from 0x5c66800 (overlap 0x1847800) to 0x748a00 2024-09-15T00:10:20.249 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:20.248+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:20.310 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:20.310 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.280509602999999, 2024-09-15T00:10:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:20.311 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:20.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:20.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:20.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.280487684999997, 2024-09-15T00:10:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:20.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:20.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.331 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.280466412999999, 2024-09-15T00:10:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.333 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:20.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:20.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.341 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.278725153000003, 2024-09-15T00:10:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.343 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:20.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:20.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:20.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.278703247000003, 2024-09-15T00:10:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:20.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:20.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.278681155000001, 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:20.359 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:20.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:20.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.278649416, 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:20.364 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:20.368 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.278351213000001, 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.369 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.278317508000001, 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:20.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.274015435000003, 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:20.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.273690565000003, 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.383 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:20.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:20.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.273280159999999, 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:20.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:20.392 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.273225257, 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:20.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:20.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.273193222000003, 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.271715391999997, 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:20.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.406 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.271693677000002, 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.407 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:20.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.271672197999997, 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:20.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.416 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.269659093999998, 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.417 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:20.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.269639453000003, 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:20.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.425 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.269592905000003, 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:20.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:20.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:20.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:20.430 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:20.431 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:20.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:20.433 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:20.429+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:20.532 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:20.531+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:20.539 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:20.539 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:20.539 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:20.539 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:20.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:20.579 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:20.596 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:20.596 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:20.613 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:20.614 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:20.630 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:20.631 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:20.633 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:20.631+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:20.649 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:20.649 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:20.666 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:20.666 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:20.669 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:20.669 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:20.695 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:20.695 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:20.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:20.745 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:20.743+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:20.752 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-15T00:10:20.835 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:20.833+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:20.867 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:20.867 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:20.876 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:20.961 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:20.959+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:20.977 INFO:teuthology.orchestra.run.smithi092.stdout:5681 write 0xd8c6000 thru 0xd8d1f94 (0xbf95 bytes) 2024-09-15T00:10:20.977 INFO:teuthology.orchestra.run.smithi092.stdout:5682 read 0x2190000 thru 0x2190212 (0x213 bytes) 2024-09-15T00:10:20.978 INFO:teuthology.orchestra.run.smithi092.stdout:5684 write 0xb4b2800 thru 0xb4b62ce (0x3acf bytes) 2024-09-15T00:10:20.989 INFO:teuthology.orchestra.run.smithi092.stdout:5685 trunc from 0xd8d2000 to 0x3dfc600 2024-09-15T00:10:21.026 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:21.026 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:21.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:21.077 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:21.075+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:21.081 INFO:teuthology.orchestra.run.smithi092.stdout:5687 write 0xc798600 thru 0xc7a223e (0x9c3f bytes) 2024-09-15T00:10:21.092 INFO:teuthology.orchestra.run.smithi092.stdout:5688 trunc from 0xc7a2400 to 0xa649400 2024-09-15T00:10:21.110 INFO:teuthology.orchestra.run.smithi092.stdout:5689 trunc from 0xa649400 to 0x3e23000 2024-09-15T00:10:21.123 INFO:teuthology.orchestra.run.smithi092.stdout:5690 read 0x3368600 thru 0x3375569 (0xcf6a bytes) 2024-09-15T00:10:21.137 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:21.136+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:21.138 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-15T00:10:21.138 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:10:21.138 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-15T00:10:21.138 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-15T00:10:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:10:21.146 INFO:teuthology.orchestra.run.smithi092.stdout:5691 write 0xbbfe200 thru 0xbc0b5e0 (0xd3e1 bytes) 2024-09-15T00:10:21.153 INFO:teuthology.orchestra.run.smithi092.stdout:5692 read 0x88a8200 thru 0x88abd23 (0x3b24 bytes) 2024-09-15T00:10:21.154 INFO:teuthology.orchestra.run.smithi092.stdout:5693 read 0x87c3800 thru 0x87c6b9d (0x339e bytes) 2024-09-15T00:10:21.154 INFO:teuthology.orchestra.run.smithi092.stdout:5695 read 0x86b9c00 thru 0x86c23af (0x87b0 bytes) 2024-09-15T00:10:21.168 INFO:teuthology.orchestra.run.smithi092.stdout:5696 write 0xe4fe400 thru 0xe501f7a (0x3b7b bytes) 2024-09-15T00:10:21.173 INFO:teuthology.orchestra.run.smithi092.stdout:5699 flatten 2024-09-15T00:10:21.184 INFO:teuthology.orchestra.run.smithi092.stdout:5700 read 0x7bc7e00 thru 0x7bd37cc (0xb9cd bytes) 2024-09-15T00:10:21.185 INFO:teuthology.orchestra.run.smithi092.stdout:5701 write 0x5deda00 thru 0x5dfba27 (0xe028 bytes) 2024-09-15T00:10:21.186 INFO:teuthology.orchestra.run.smithi092.stdout:5704 write 0x71d3800 thru 0x71d7ede (0x46df bytes) 2024-09-15T00:10:21.187 INFO:teuthology.orchestra.run.smithi092.stdout:5705 punch from 0x7b60400 to 0x7b65400, (0x5000 bytes) 2024-09-15T00:10:21.187 INFO:teuthology.orchestra.run.smithi092.stdout:5709 write 0x79d2a00 thru 0x79e01f1 (0xd7f2 bytes) 2024-09-15T00:10:21.188 INFO:teuthology.orchestra.run.smithi092.stdout:5710 read 0x36fe800 thru 0x3700ac8 (0x22c9 bytes) 2024-09-15T00:10:21.189 INFO:teuthology.orchestra.run.smithi092.stdout:5716 punch from 0x226f400 to 0x2277c00, (0x8800 bytes) 2024-09-15T00:10:21.189 INFO:teuthology.orchestra.run.smithi092.stdout:5718 read 0x2bb5000 thru 0x2bbb64d (0x664e bytes) 2024-09-15T00:10:21.190 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:21.190 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:21.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:21.201 INFO:teuthology.orchestra.run.smithi092.stdout:5719 trunc from 0xe502000 to 0x4c77000 2024-09-15T00:10:21.233 INFO:teuthology.orchestra.run.smithi092.stdout:5720 trunc from 0x4c77000 to 0x875a400 2024-09-15T00:10:21.240 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:21.239+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:21.246 INFO:teuthology.orchestra.run.smithi092.stdout:5722 punch from 0x3f86000 to 0x3f8e600, (0x8600 bytes) 2024-09-15T00:10:21.246 INFO:teuthology.orchestra.run.smithi092.stdout:5723 write 0x62de600 thru 0x62e0e93 (0x2894 bytes) 2024-09-15T00:10:21.247 INFO:teuthology.orchestra.run.smithi092.stdout:5726 read 0x2a59800 thru 0x2a61986 (0x8187 bytes) 2024-09-15T00:10:21.247 INFO:teuthology.orchestra.run.smithi092.stdout:5730 write 0x5281a00 thru 0x5285dd8 (0x43d9 bytes) 2024-09-15T00:10:21.248 INFO:teuthology.orchestra.run.smithi092.stdout:5731 write 0xac1c00 thru 0xad13a8 (0xf7a9 bytes) 2024-09-15T00:10:21.248 INFO:teuthology.orchestra.run.smithi092.stdout:5732 read 0x2584800 thru 0x258bc28 (0x7429 bytes) 2024-09-15T00:10:21.251 INFO:teuthology.orchestra.run.smithi092.stdout:5733 trunc from 0x875a400 to 0x6c07400 2024-09-15T00:10:21.285 INFO:teuthology.orchestra.run.smithi092.stdout:5735 write 0xd7cfc00 thru 0xd7da727 (0xab28 bytes) 2024-09-15T00:10:21.286 INFO:teuthology.orchestra.run.smithi092.stdout:5737 write 0x8c63e00 thru 0x8c687a2 (0x49a3 bytes) 2024-09-15T00:10:21.288 INFO:teuthology.orchestra.run.smithi092.stdout:5738 trunc from 0xd7da800 to 0xeb25600 2024-09-15T00:10:21.304 INFO:teuthology.orchestra.run.smithi092.stdout:5739 punch from 0x8966200 to 0x8971800, (0xb600 bytes) 2024-09-15T00:10:21.304 INFO:teuthology.orchestra.run.smithi092.stdout:5740 punch from 0x7ba9a00 to 0x7bb7c00, (0xe200 bytes) 2024-09-15T00:10:21.304 INFO:teuthology.orchestra.run.smithi092.stdout:5742 punch from 0x681d200 to 0x6823400, (0x6200 bytes) 2024-09-15T00:10:21.317 INFO:teuthology.orchestra.run.smithi092.stdout:5747 trunc from 0xeb25600 to 0x36be600 2024-09-15T00:10:21.339 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:21.338+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:21.349 INFO:teuthology.orchestra.run.smithi092.stdout:5748 trunc from 0x36be600 to 0x87ede00 2024-09-15T00:10:21.354 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:21.354 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:21.357 INFO:teuthology.orchestra.run.smithi092.stdout:5749 punch from 0x858e200 to 0x859b000, (0xce00 bytes) 2024-09-15T00:10:21.358 INFO:teuthology.orchestra.run.smithi092.stdout:5751 write 0x41ed800 thru 0x41f2d7d (0x557e bytes) 2024-09-15T00:10:21.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:21.365 INFO:teuthology.orchestra.run.smithi092.stdout:5752 trunc from 0x87ede00 to 0xe4d1e00 2024-09-15T00:10:21.391 INFO:teuthology.orchestra.run.smithi092.stdout:5754 trunc from 0xe4d1e00 to 0x2b8f000 2024-09-15T00:10:21.415 INFO:teuthology.orchestra.run.smithi092.stdout:5755 read 0xa60200 thru 0xa69805 (0x9606 bytes) 2024-09-15T00:10:21.422 INFO:teuthology.orchestra.run.smithi092.stdout:5756 read 0x12fa600 thru 0x1306d51 (0xc752 bytes) 2024-09-15T00:10:21.427 INFO:teuthology.orchestra.run.smithi092.stdout:5758 trunc from 0x2b8f000 to 0x66b0000 2024-09-15T00:10:21.437 INFO:teuthology.orchestra.run.smithi092.stdout:5760 trunc from 0x66b0000 to 0x598da00 2024-09-15T00:10:21.447 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-15T00:10:21.445+0000 7f73f660c640 -1 mon.a@0(leader).osd e83 definitely_dead 0 2024-09-15T00:10:21.448 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:21.447+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:21.465 INFO:teuthology.orchestra.run.smithi092.stdout:5761 write 0xbb3a600 thru 0xbb3c13c (0x1b3d bytes) 2024-09-15T00:10:21.465 INFO:teuthology.orchestra.run.smithi092.stdout:5762 write 0xae47e00 thru 0xae49997 (0x1b98 bytes) 2024-09-15T00:10:21.465 INFO:teuthology.orchestra.run.smithi092.stdout:5763 read 0x6992400 thru 0x6993e2e (0x1a2f bytes) 2024-09-15T00:10:21.465 INFO:teuthology.orchestra.run.smithi092.stdout:5764 write 0x77dae00 thru 0x77e8ac0 (0xdcc1 bytes) 2024-09-15T00:10:21.482 INFO:teuthology.orchestra.run.smithi092.stdout:5766 write 0xe934600 thru 0xe93ded8 (0x98d9 bytes) 2024-09-15T00:10:21.487 INFO:teuthology.orchestra.run.smithi092.stdout:5769 read 0xa067a00 thru 0xa07210c (0xa70d bytes) 2024-09-15T00:10:21.488 INFO:teuthology.orchestra.run.smithi092.stdout:5770 write 0x5bdd000 thru 0x5be7805 (0xa806 bytes) 2024-09-15T00:10:21.491 INFO:teuthology.orchestra.run.smithi092.stdout:5771 read 0xcd66800 thru 0xcd68587 (0x1d88 bytes) 2024-09-15T00:10:21.491 INFO:teuthology.orchestra.run.smithi092.stdout:5774 read 0x690d000 thru 0x6910de2 (0x3de3 bytes) 2024-09-15T00:10:21.492 INFO:teuthology.orchestra.run.smithi092.stdout:5775 write 0xa965e00 thru 0xa96f214 (0x9415 bytes) 2024-09-15T00:10:21.493 INFO:teuthology.orchestra.run.smithi092.stdout:5778 read 0x684a00 thru 0x69169c (0xcc9d bytes) 2024-09-15T00:10:21.494 INFO:teuthology.orchestra.run.smithi092.stdout:5780 read 0x95c0e00 thru 0x95c2b0f (0x1d10 bytes) 2024-09-15T00:10:21.494 INFO:teuthology.orchestra.run.smithi092.stdout:5781 read 0x9aa7200 thru 0x9ab122f (0xa030 bytes) 2024-09-15T00:10:21.494 INFO:teuthology.orchestra.run.smithi092.stdout:5782 write 0x8c1ac00 thru 0x8c202de (0x56df bytes) 2024-09-15T00:10:21.494 INFO:teuthology.orchestra.run.smithi092.stdout:5783 read 0x8c90a00 thru 0x8c96775 (0x5d76 bytes) 2024-09-15T00:10:21.494 INFO:teuthology.orchestra.run.smithi092.stdout:5785 write 0xf3b000 thru 0xf457db (0xa7dc bytes) 2024-09-15T00:10:21.495 INFO:teuthology.orchestra.run.smithi092.stdout:5786 read 0xeb400 thru 0xf2e76 (0x7a77 bytes) 2024-09-15T00:10:21.495 INFO:teuthology.orchestra.run.smithi092.stdout:5799 punch from 0x9b97000 to 0x9b99e00, (0x2e00 bytes) 2024-09-15T00:10:21.496 INFO:teuthology.orchestra.run.smithi092.stdout:5800 punch from 0x3854a00 to 0x3856e00, (0x2400 bytes) 2024-09-15T00:10:21.496 INFO:teuthology.orchestra.run.smithi092.stdout:5803 read 0xccb2c00 thru 0xccbea10 (0xbe11 bytes) 2024-09-15T00:10:21.498 INFO:teuthology.orchestra.run.smithi092.stdout:5806 write 0x669b000 thru 0x66a7ab1 (0xcab2 bytes) 2024-09-15T00:10:21.499 INFO:teuthology.orchestra.run.smithi092.stdout:5807 write 0x2b94e00 thru 0x2b986bc (0x38bd bytes) 2024-09-15T00:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout:5808 write 0x4d15a00 thru 0x4d23766 (0xdd67 bytes) 2024-09-15T00:10:21.501 INFO:teuthology.orchestra.run.smithi092.stdout:5814 read 0x6445a00 thru 0x6454dfc (0xf3fd bytes) 2024-09-15T00:10:21.511 INFO:teuthology.orchestra.run.smithi092.stdout:5817 trunc from 0xe93e000 to 0x715b600 2024-09-15T00:10:21.524 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:21.524 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:21.533 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:21.534 INFO:teuthology.orchestra.run.smithi092.stdout:5819 trunc from 0x715b600 to 0xc0f0c00 2024-09-15T00:10:21.541 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:21.539+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:21.548 INFO:teuthology.orchestra.run.smithi092.stdout:5820 read 0x7d03a00 thru 0x7d11aca (0xe0cb bytes) 2024-09-15T00:10:21.552 INFO:teuthology.orchestra.run.smithi092.stdout:5822 trunc from 0xc0f0c00 to 0x8dbee00 2024-09-15T00:10:21.567 INFO:teuthology.orchestra.run.smithi092.stdout:5823 read 0x956000 thru 0x963725 (0xd726 bytes) 2024-09-15T00:10:21.570 INFO:teuthology.orchestra.run.smithi092.stdout:5824 trunc from 0x8dbee00 to 0x61d3e00 2024-09-15T00:10:21.581 INFO:teuthology.orchestra.run.smithi092.stdout:5825 read 0x252ee00 thru 0x2536ab9 (0x7cba bytes) 2024-09-15T00:10:21.582 INFO:teuthology.orchestra.run.smithi092.stdout:5826 clone 38 order 22 su 524288 sc 6 2024-09-15T00:10:21.642 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:21.640+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:21.675 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:21.675 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:21.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:21.742 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:21.741+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:21.788 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:21.788 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:21.797 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:21.844 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:21.842+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:21.904 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:21.904 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:21.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:21.954 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:21.952+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:22.045 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:22.043+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:22.060 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:22.060 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:22.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:22.137 INFO:teuthology.orchestra.run.smithi027.stderr:marked out osd.0. 2024-09-15T00:10:22.145 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:22.143+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:22.219 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:22.219 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:22.228 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:22.246 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:22.245+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:22.335 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:22.336 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:22.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:22.346 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:22.345+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:22.447 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:22.446+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:22.451 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:22.451 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:22.460 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:22.549 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:22.547+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:22.567 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:22.567 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:22.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:22.650 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:22.648+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:22.670 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:22.685 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:22.685 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:22.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:22.750 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:22.748+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:22.775 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:22.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:22.792 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:22.809 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:22.809 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:22.827 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:22.827 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:22.838 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:22.839 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:22.844 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:22.845 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:22.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:22.851 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:22.850+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:22.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:22.861 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:22.864 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:22.864 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:22.952 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:22.950+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:22.953 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0390996900000005, 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0390793840000008, 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0390574440000009, 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0371996729999999, 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0371699939999992, 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0371469009999998, 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0371217280000007, 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0371010869999999, 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0370805460000003, 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0370570640000008, 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0351934079999996, 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0351636390000003, 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0351414600000002, 2024-09-15T00:10:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0351142099999997, 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0350876160000002, 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0350682829999993, 2024-09-15T00:10:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0330198740000007, 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0330046409999998, 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.2677281220000003, 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:10:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:10:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:10:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.2216374520000004, 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:10:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:10:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:10:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.049 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:23.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:23.053 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:23.051+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:23.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:10:23.154 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:23.152+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:23.198 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:20492 3.1b 3.4083537b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:23.152190+0000", 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.045455951000000001, 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045476470999999997, 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20492 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152190+0000", 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152190+0000", 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152192+0000", 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.409e-06 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152197+0000", 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4620000000000004e-06 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152215+0000", 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8456000000000001e-05 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152224+0000", 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8519999999999993e-06 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152315+0000", 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0599000000000005e-05 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-15T00:10:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152324+0000", 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4010000000000003e-06 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5575.0:1746 3.1b 3.4083537b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:23.152242+0000", 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.045403856999999999, 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.04546443, 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5575", 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2000336920", 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1746 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152242+0000", 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152242+0000", 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152243+0000", 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152245+0000", 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3599999999999999e-06 2024-09-15T00:10:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152252+0000", 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8480000000000003e-06 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152257+0000", 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3530000000000002e-06 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152348+0000", 2024-09-15T00:10:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1154000000000002e-05 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:23.152354+0000", 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8069999999999996e-06 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 2 2024-09-15T00:10:23.207 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:23.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:23.294 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:23.292+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:23.350 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:23.351 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:23.355 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:23.353+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:23.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:23.456 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:23.454+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:23.504 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1772 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5373952~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.383138+0000", 2024-09-15T00:10:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1182704999999995, 2024-09-15T00:10:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044949414, 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1772 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383138+0000", 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383138+0000", 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383138+0000", 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383140+0000", 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.951e-06 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383147+0000", 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4999999999999996e-06 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383152+0000", 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8930000000000004e-06 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.427906+0000", 2024-09-15T00:10:23.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044753715999999999 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.427993+0000", 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7021000000000006e-05 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.428087+0000", 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4938000000000005e-05 2024-09-15T00:10:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1773 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5414912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.383158+0000", 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.118250626, 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045162009000000003, 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1773 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383158+0000", 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.383158+0000", 2024-09-15T00:10:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.384928+0000", 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001769863 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.384932+0000", 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7230000000000002e-06 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.384941+0000", 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6609999999999995e-06 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.384947+0000", 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.268e-06 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.428132+0000", 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043184563000000002 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.428216+0000", 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4244000000000002e-05 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.428320+0000", 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103687 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1781 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5742592~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385112+0000", 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1162968949999996, 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044888666000000001, 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1781 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385112+0000", 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385112+0000", 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385112+0000", 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385114+0000", 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.84e-06 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385120+0000", 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9070000000000004e-06 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385124+0000", 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5770000000000004e-06 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.429881+0000", 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044756706 2024-09-15T00:10:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.429945+0000", 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4177000000000006e-05 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430000+0000", 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5021000000000001e-05 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1782 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5783552~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385130+0000", 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1162788339999992, 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045117168999999999, 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1782 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385130+0000", 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385130+0000", 2024-09-15T00:10:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385130+0000", 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.65e-07 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385132+0000", 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7549999999999999e-06 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385138+0000", 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6699999999999999e-06 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385142+0000", 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6299999999999997e-06 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430043+0000", 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044900936000000002 2024-09-15T00:10:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430143+0000", 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010037799999999999 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430247+0000", 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103435 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1783 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5824512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385147+0000", 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1162613300000004, 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045261416999999998, 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1783 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385147+0000", 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385147+0000", 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385148+0000", 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385149+0000", 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6780000000000001e-06 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385155+0000", 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5829999999999997e-06 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385159+0000", 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4449999999999998e-06 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430262+0000", 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045102237000000003 2024-09-15T00:10:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430326+0000", 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4715999999999995e-05 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430409+0000", 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2415000000000003e-05 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1784 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5865472~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385164+0000", 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1162442180000003, 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045458331999999997, 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1784 2024-09-15T00:10:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385164+0000", 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385164+0000", 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385165+0000", 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3799999999999998e-07 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385166+0000", 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.685e-06 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385172+0000", 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5609999999999998e-06 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385176+0000", 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4660000000000001e-06 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430453+0000", 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045276755000000002 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430540+0000", 2024-09-15T00:10:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430623+0000", 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2479999999999996e-05 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1785 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5906432~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385181+0000", 2024-09-15T00:10:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1162271599999993, 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045656909000000002, 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1785 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385181+0000", 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385181+0000", 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385182+0000", 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385183+0000", 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.748e-06 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385189+0000", 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1279999999999999e-06 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385193+0000", 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.493e-06 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430669+0000", 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045475504 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430752+0000", 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3548999999999996e-05 2024-09-15T00:10:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430838+0000", 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6143999999999994e-05 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1786 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5947392~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385206+0000", 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1162021160000002, 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045836012000000002, 2024-09-15T00:10:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1786 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385206+0000", 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385206+0000", 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385207+0000", 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3900000000000005e-07 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385209+0000", 2024-09-15T00:10:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9769999999999999e-06 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385215+0000", 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6110000000000001e-06 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385219+0000", 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4290000000000003e-06 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430875+0000", 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045655880000000003 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.430954+0000", 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9406000000000005e-05 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431042+0000", 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8170000000000005e-05 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1787 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 5988352~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385225+0000", 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1161833699999999, 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045998764999999997, 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1787 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385225+0000", 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385225+0000", 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385225+0000", 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0899999999999997e-07 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385227+0000", 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.962e-06 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385235+0000", 2024-09-15T00:10:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6669999999999996e-06 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385240+0000", 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.95e-06 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431073+0000", 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045832497999999999 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431144+0000", 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1464999999999994e-05 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431224+0000", 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9913999999999994e-05 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1788 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 6029312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385247+0000", 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1161619789999992, 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046194975999999999, 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1788 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385247+0000", 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385247+0000", 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385247+0000", 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385249+0000", 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.995e-06 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385255+0000", 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385260+0000", 2024-09-15T00:10:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431266+0000", 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046005742000000002 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431349+0000", 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2737000000000004e-05 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431442+0000", 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2927e-05 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1789 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 6070272~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.385266+0000", 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1161424990000004, 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046303922999999997, 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1789 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385266+0000", 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.385266+0000", 2024-09-15T00:10:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.386929+0000", 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016625030000000001 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.386933+0000", 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3880000000000002e-06 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.386945+0000", 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620999999999999e-05 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.386952+0000", 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5800000000000003e-06 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431477+0000", 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044525153999999997 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431519+0000", 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347000000000002e-05 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431570+0000", 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1329999999999998e-05 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1792 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 6193152~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387027+0000", 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1143820200000008, 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045050564000000001, 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1792 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387027+0000", 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387027+0000", 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387027+0000", 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-07 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387029+0000", 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387037+0000", 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4309999999999997e-06 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387049+0000", 2024-09-15T00:10:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2468e-05 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431901+0000", 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044851374999999999 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.431991+0000", 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0056999999999995e-05 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432077+0000", 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6379999999999996e-05 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1793 3.9 3:96d993ec:::rbd_data.1129696a4676.0000000000000004:24 [read 6234112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387058+0000", 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1143506310000006, 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045238681000000003, 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1793 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387058+0000", 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387058+0000", 2024-09-15T00:10:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387058+0000", 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0200000000000002e-07 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387061+0000", 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7379999999999999e-06 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387069+0000", 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1899999999999995e-06 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387075+0000", 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3499999999999996e-06 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432110+0000", 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045035155 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432188+0000", 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8187000000000002e-05 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432297+0000", 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108559 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1794 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387097+0000", 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1143115009999995, 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241772999999999, 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1794 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387097+0000", 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387097+0000", 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387098+0000", 2024-09-15T00:10:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4099999999999999e-07 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387100+0000", 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.39e-06 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387107+0000", 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1600000000000001e-06 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387113+0000", 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4829999999999999e-06 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432330+0000", 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045217368000000001 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432339+0000", 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8310000000000007e-06 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1795 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387120+0000", 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1142890130000005, 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045243522000000001, 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1795 2024-09-15T00:10:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387120+0000", 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387120+0000", 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387120+0000", 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6699999999999999e-07 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387122+0000", 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0779999999999998e-06 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387128+0000", 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0680000000000002e-06 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387133+0000", 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.797e-06 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432359+0000", 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045225564000000003 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432363+0000", 2024-09-15T00:10:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5480000000000003e-06 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1796 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387140+0000", 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1142685179999994, 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045255984999999999, 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1796 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387140+0000", 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387140+0000", 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387140+0000", 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387143+0000", 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387149+0000", 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7259999999999996e-06 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387154+0000", 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0350000000000001e-06 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432388+0000", 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045234116999999997 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432396+0000", 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1797 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387161+0000", 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1142473969999998, 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045252725000000001, 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1797 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387161+0000", 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387161+0000", 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387162+0000", 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387164+0000", 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-15T00:10:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387170+0000", 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3049999999999998e-06 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387175+0000", 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8489999999999997e-06 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432408+0000", 2024-09-15T00:10:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045233425000000001 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432414+0000", 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7119999999999997e-06 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1798 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387180+0000", 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1142285790000006, 2024-09-15T00:10:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045265596999999998, 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1798 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387180+0000", 2024-09-15T00:10:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387180+0000", 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387180+0000", 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387182+0000", 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.804e-06 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387188+0000", 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6919999999999999e-06 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387192+0000", 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.639e-06 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432437+0000", 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045244541999999999 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432446+0000", 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5599999999999994e-06 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1799 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387198+0000", 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1142101740000001, 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045270091999999998, 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1799 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387198+0000", 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387198+0000", 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387199+0000", 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7500000000000001e-07 2024-09-15T00:10:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387201+0000", 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.928e-06 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387206+0000", 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6409999999999999e-06 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387211+0000", 2024-09-15T00:10:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7600000000000002e-06 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432464+0000", 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045253267999999999 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432468+0000", 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1200000000000004e-06 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:1800 3.9 3.37c99b69 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.387216+0000", 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.1141923210000009, 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045283179, 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1800 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387216+0000", 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.387216+0000", 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.388928+0000", 2024-09-15T00:10:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001712019 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.388933+0000", 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2910000000000001e-06 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.388944+0000", 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1834000000000001e-05 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.388955+0000", 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205e-05 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432492+0000", 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043537449999999998 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.432499+0000", 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3799999999999996e-06 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:23.596 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:23.597 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:23.598 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:23.596+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:23.698 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:23.697+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:23.741 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:23.741 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:23.741 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:23.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:23.833 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:23.832+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:23.892 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:23.892 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:23.892 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:23.892 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:23.892 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:23.900 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:23.898+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:23.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:24.000 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:23.999+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:24.046 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:24.046 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:24.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.161370470999998, 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.047 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:24.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.051 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.161331841000006, 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:24.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.161305904000002, 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:24.056 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.161289835999995, 2024-09-15T00:10:24.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:24.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.161148741000005, 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.066 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:24.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:24.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.160291955000005, 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.071 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.160256939000007, 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.076 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:24.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:24.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.160231335999995, 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:24.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.160057069999993, 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.085 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:24.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.089 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.159825931, 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.090 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:24.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.094 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.159013986999994, 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:24.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:24.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.158934247000005, 2024-09-15T00:10:24.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:24.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:24.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.158729222000005, 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:24.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.108 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.158517689000007, 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.109 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.113 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.155961681999997, 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:24.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.153978576, 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:24.118 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:24.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.153959005000004, 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:24.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:24.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.883216155, 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.128 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:24.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:24.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9774646600000008, 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.134 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:24.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3603772200000002, 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:24.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:24.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:24.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:24.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:24.144 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:24.146 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:24.144+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:24.146 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:24.147 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:10:24.246 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:24.244+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:24.302 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5575.0:1792 3.b 3:d5b7b4ec:::rbd_data.11291eb5e35f.0000000000000000:head [read 8724480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:24.301234+0000", 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.000100574, 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118887, 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5575", 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2000336920", 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1792 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:24.301234+0000", 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:24.301234+0000", 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:24.301235+0000", 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2799999999999996e-07 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:24.301239+0000", 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.177e-06 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:24.301250+0000", 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0941000000000001e-05 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:24.301258+0000", 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2190000000000004e-06 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:24.301296+0000", 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8003000000000002e-05 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:24.301347+0000", 2024-09-15T00:10:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0358999999999997e-05 2024-09-15T00:10:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 1 2024-09-15T00:10:24.307 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:24.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:24.347 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:24.346+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:24.449 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:24.447+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:24.479 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:24.480 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:24.490 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:24.550 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:24.548+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:24.616 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585649166000003, 2024-09-15T00:10:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585628911000001, 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585609165000001, 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585589214999999, 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585561714999997, 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585544874, 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585528875999998, 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585505888, 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585480003999997, 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585463576999999, 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585449306000001, 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.585439989000001, 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.675 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.583770977, 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:24.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:24.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.583735914000002, 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.583710887000002, 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:24.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:24.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:24.690 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.583686579000002, 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.696 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.581494327999998, 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:10:24.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:10:24.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.581470195999998, 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:24.701 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:24.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:24.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.581440823999998, 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:24.706 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:24.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.9169373969999999, 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:24.711 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:10:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:10:24.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:24.717 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:24.718 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-15T00:10:24.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:24.719 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:24.716+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:24.732 INFO:teuthology.orchestra.run.smithi092.stdout:5827 trunc from 0x61d3e00 to 0x9146800 2024-09-15T00:10:24.758 INFO:teuthology.orchestra.run.smithi092.stdout:5831 trunc from 0x9146800 to 0xd4b7000 2024-09-15T00:10:24.768 INFO:teuthology.orchestra.run.smithi092.stdout:5832 read 0xc029e00 thru 0xc03266f (0x8870 bytes) 2024-09-15T00:10:24.783 INFO:teuthology.orchestra.run.smithi092.stdout:5834 trunc from 0xd4b7000 to 0xc9a800 2024-09-15T00:10:24.819 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:24.817+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:24.838 INFO:teuthology.orchestra.run.smithi092.stdout:5835 write 0xedc8800 thru 0xedcac5a (0x245b bytes) 2024-09-15T00:10:24.839 INFO:teuthology.orchestra.run.smithi092.stdout:5836 flatten 2024-09-15T00:10:24.855 INFO:teuthology.orchestra.run.smithi092.stdout:5838 punch from 0xaa1ba00 to 0xaa21e00, (0x6400 bytes) 2024-09-15T00:10:24.855 INFO:teuthology.orchestra.run.smithi092.stdout:5839 punch from 0xe121600 to 0xe12ca00, (0xb400 bytes) 2024-09-15T00:10:24.855 INFO:teuthology.orchestra.run.smithi092.stdout:5840 read 0xa3eb800 thru 0xa3ec159 (0x95a bytes) 2024-09-15T00:10:24.856 INFO:teuthology.orchestra.run.smithi092.stdout:5841 read 0xde5a800 thru 0xde65594 (0xad95 bytes) 2024-09-15T00:10:24.856 INFO:teuthology.orchestra.run.smithi092.stdout:5843 clone 39 order 20 su 262144 sc 7 2024-09-15T00:10:24.865 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:24.894 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:24.894 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:24.895 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:24.895 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:24.904 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:24.921 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:24.920+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:24.977 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:24.994 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:25.011 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:25.012 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:25.022 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:25.020+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:25.028 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:25.028 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:25.045 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:25.046 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:10:25.053 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:25.053 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:25.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:25.062 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:25.063 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:25.065 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:25.065 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:25.123 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:25.122+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:25.172 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:25.173 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:25.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:25.223 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:25.221+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:25.284 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:25.284 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:25.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:25.336 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:25.334+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:25.394 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:25.394 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:25.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:25.445 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:25.443+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:25.525 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:25.523+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:25.546 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:25.546 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:25.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:25.640 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:25.638+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:25.699 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:25.699 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:25.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:10:25.726 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:25.725+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:25.828 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:25.825+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:25.849 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:25.849 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:25.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:10:25.929 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:25.927+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:25.959 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:25.959 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:25.967 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:26.029 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:26.027+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:26.110 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:26.111 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:26.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:26.163 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:26.161+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:26.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:10:26.231 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:26.232 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:26.232 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:26.230+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:26.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:26.333 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:26.331+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:26.386 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:26.386 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:26.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:26.438 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:26.436+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:26.505 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:26.505 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:26.514 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:26.555 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:26.553+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:26.593 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-15T00:10:26.635 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:26.633+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:26.662 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:26.662 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:26.671 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:26.737 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:26.736+0000 7f149e473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32255) UID: 0 2024-09-15T00:10:26.822 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:26.823 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:26.832 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:26.876 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:26.874+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:26.888 INFO:teuthology.orchestra.run.smithi092.stdout:5845 write 0x24ab800 thru 0x24b4318 (0x8b19 bytes) 2024-09-15T00:10:26.889 INFO:teuthology.orchestra.run.smithi092.stdout:5849 write 0xc92fe00 thru 0xc93f10e (0xf30f bytes) 2024-09-15T00:10:26.889 INFO:teuthology.orchestra.run.smithi092.stdout:5850 write 0x58ca00 thru 0x5996ea (0xcceb bytes) 2024-09-15T00:10:26.904 INFO:teuthology.orchestra.run.smithi092.stdout:5851 trunc from 0xedcae00 to 0x3463400 2024-09-15T00:10:26.939 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:26.937+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:26.942 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:26.942 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:26.951 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:26.959 INFO:teuthology.orchestra.run.smithi092.stdout:5852 write 0x3526400 thru 0x3531d9f (0xb9a0 bytes) 2024-09-15T00:10:26.965 INFO:teuthology.orchestra.run.smithi092.stdout:5853 write 0x2057c00 thru 0x2066d75 (0xf176 bytes) 2024-09-15T00:10:26.965 INFO:teuthology.orchestra.run.smithi092.stdout:5854 read 0x21a7a00 thru 0x21ac6a8 (0x4ca9 bytes) 2024-09-15T00:10:26.986 INFO:teuthology.orchestra.run.smithi092.stdout:5855 write 0x8228600 thru 0x822decd (0x58ce bytes) 2024-09-15T00:10:26.987 INFO:teuthology.orchestra.run.smithi092.stdout:5860 write 0x34dc600 thru 0x34ebc6e (0xf66f bytes) 2024-09-15T00:10:26.987 INFO:teuthology.orchestra.run.smithi092.stdout:5862 punch from 0x667de00 to 0x6687800, (0x9a00 bytes) 2024-09-15T00:10:27.002 INFO:teuthology.orchestra.run.smithi092.stdout:5863 write 0xd449400 thru 0xd452b4a (0x974b bytes) 2024-09-15T00:10:27.003 INFO:teuthology.orchestra.run.smithi092.stdout:5864 write 0x3bf9c00 thru 0x3c06626 (0xca27 bytes) 2024-09-15T00:10:27.003 INFO:teuthology.orchestra.run.smithi092.stdout:5865 write 0x4032800 thru 0x403f8db (0xd0dc bytes) 2024-09-15T00:10:27.003 INFO:teuthology.orchestra.run.smithi092.stdout:5866 read 0xd2a9a00 thru 0xd2aa222 (0x823 bytes) 2024-09-15T00:10:27.003 INFO:teuthology.orchestra.run.smithi092.stdout:5869 punch from 0x787800 to 0x793a00, (0xc200 bytes) 2024-09-15T00:10:27.019 INFO:teuthology.orchestra.run.smithi092.stdout:5871 trunc from 0xd452c00 to 0x41a800 2024-09-15T00:10:27.040 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:27.038+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:27.062 INFO:teuthology.orchestra.run.smithi092.stdout:5872 write 0x216b800 thru 0x2177039 (0xb83a bytes) 2024-09-15T00:10:27.066 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:27.091 INFO:teuthology.orchestra.run.smithi092.stdout:5873 write 0xe378400 thru 0xe37f8a6 (0x74a7 bytes) 2024-09-15T00:10:27.100 INFO:teuthology.orchestra.run.smithi092.stdout:5874 trunc from 0xe37fa00 to 0x64d9a00 2024-09-15T00:10:27.113 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:27.113 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:27.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:27.125 INFO:teuthology.orchestra.run.smithi092.stdout:5875 trunc from 0x64d9a00 to 0xaebc00 2024-09-15T00:10:27.139 INFO:teuthology.orchestra.run.smithi092.stdout:5876 punch from 0x8b7a00 to 0x8bd000, (0x5600 bytes) 2024-09-15T00:10:27.148 INFO:teuthology.orchestra.run.smithi092.stdout:5879 read 0x961a00 thru 0x970d39 (0xf33a bytes) 2024-09-15T00:10:27.153 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-15T00:10:27.153 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:10:27.154 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [1, 2, 3, 4, 5] 2024-09-15T00:10:27.154 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T00:10:27.154 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:10:27.164 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:10:27.164 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:27.162+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:27.182 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:27.185 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T00:10:27.185 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-15T00:10:27.185 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi079.front.sepia.ceph.com 2024-09-15T00:10:27.186 INFO:tasks.thrashosds.thrasher:remote for exp osd 0 is ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:10:27.186 DEBUG:teuthology.orchestra.run.smithi027:> type ceph-objectstore-tool 2024-09-15T00:10:27.199 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:27.200 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:27.200 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T00:10:27.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-15T00:10:27.217 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:27.218 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:27.236 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:27.236 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:27.240 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:27.240 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.212541303000002, 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.241 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.212519385, 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.246 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:27.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.250 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.212498113000002, 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:27.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.210756852999999, 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:27.255 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.259 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.210734946999999, 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.260 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.264 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.210712854999997, 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:27.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:27.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.210681116000003, 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:27.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:27.273 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.210382912999997, 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.274 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:27.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:27.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:27.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:27.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.210349207999997, 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.279 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:27.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.206047134999999, 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:27.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:27.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:27.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:27.288 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.205722264999999, 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:27.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:27.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.205311860000002, 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.293 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:27.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:27.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.205256957000003, 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.298 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.302 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.205224921999999, 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.203747092, 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:27.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.203725376999998, 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:27.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.313 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:27.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:27.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:27.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.203703898000001, 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:27.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:27.321 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.201690794000001, 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.322 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.326 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.201671152999999, 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:27.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:27.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.201624604999999, 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:27.331 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:27.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:27.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:27.335 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:27.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:27.337 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:27.336+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:27.438 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:27.436+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:27.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:27.538 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:27.536+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:27.592 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:27.592 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:27.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:27.639 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:27.638+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:27.740 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:27.737+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:27.745 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.830556744999999, 2024-09-15T00:10:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:27.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:27.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.830536438999999, 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:27.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:27.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.830514499, 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:27.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.828656728, 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.760 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.828627049, 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:27.764 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:27.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.828603956, 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:27.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:27.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:27.773 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.828578782999999, 2024-09-15T00:10:27.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:27.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.828558142, 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.828537601000001, 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.786 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:27.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.828514118999999, 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:27.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.826650463, 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:27.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:27.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.826620694000001, 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:27.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.826598515000001, 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:27.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:27.809 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.826571265, 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.826544671000001, 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:27.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.826525338, 2024-09-15T00:10:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:27.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:27.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.824476928999999, 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:27.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.824461696, 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:27.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0591851769999998, 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:27.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:10:27.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0130945069999999, 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:10:27.839 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:27.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:27.844 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:27.845 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:27.846 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:27.847 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:27.844+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:27.946 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:27.944+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:27.990 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:27.990 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:27.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:28.047 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:28.045+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:28.100 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:28.101 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:28.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:28.148 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:28.146+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:28.189 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:28.192 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:28.193 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:28.215 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:28.215 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:28.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.330296179000001, 2024-09-15T00:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:28.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:28.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:28.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:28.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.330257548999995, 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.221 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.330231612000006, 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.330215543999998, 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.330074448999994, 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:28.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.329217662999994, 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.240 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:28.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:28.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:28.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:28.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:28.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.244 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.329182646999996, 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.245 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:28.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.250 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.250 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:28.250 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.329157043999999, 2024-09-15T00:10:28.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:28.250 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:28.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:28.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:28.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:28.256 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.328982777999997, 2024-09-15T00:10:28.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:28.256 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.256 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:28.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.260 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.328751639000004, 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.327939694999998, 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.265 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.327859954999994, 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:28.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.327654929999994, 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:28.274 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:28.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:28.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:28.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:28.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.327443396999996, 2024-09-15T00:10:28.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:28.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:28.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.324887390000001, 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.322904284000003, 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.322884712999993, 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:28.293 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:28.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.052141862999999, 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:28.298 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:28.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:28.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.146390368, 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:28.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:28.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.5293029279999999, 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:28.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:28.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:28.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:28.312 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:28.314 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:28.311+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:28.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:28.415 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:28.412+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:28.424 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5644.0:745 3.1a 3.ac0e573a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.248250103, 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.24827318200000001, 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "waiting for peered", 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5644", 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1022470035", 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 745 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.425 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171868+0000", 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-06 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171876+0000", 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.593e-06 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171890+0000", 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4347e-05 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171897+0000", 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8449999999999997e-06 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171949+0000", 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2206000000000001e-05 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171959+0000", 2024-09-15T00:10:28.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7829999999999993e-06 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20982 3.1a 3.ac0e573a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.24818241299999999, 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.248248317, 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "waiting for peered", 2024-09-15T00:10:28.429 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20982 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171935+0000", 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1200000000000003e-07 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171938+0000", 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9550000000000001e-06 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171944+0000", 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171948+0000", 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8519999999999997e-06 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171977+0000", 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9539999999999998e-05 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171981+0000", 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.478e-06 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 2 2024-09-15T00:10:28.433 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:28.435 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:28.516 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:28.513+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:28.542 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:28.542 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:28.542 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:28.542 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:28.542 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:28.550 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:28.616 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:28.613+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:28.700 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2657 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2932736~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670220968000002, 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059970783999999999, 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2657 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.701 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027561+0000", 2024-09-15T00:10:28.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:28.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027563+0000", 2024-09-15T00:10:28.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0499999999999999e-06 2024-09-15T00:10:28.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027569+0000", 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9970000000000004e-06 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027574+0000", 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087324+0000", 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059749947999999997 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087430+0000", 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105788 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087532+0000", 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102086 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670200713, 2024-09-15T00:10:28.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.707 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:28.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670180967, 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.712 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670161016999998, 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:28.717 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:28.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:28.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670133516999996, 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:28.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.726 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670116675999999, 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:28.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670100677999997, 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.731 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:28.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.735 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670077689999999, 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.736 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670051806000004, 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.741 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:28.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670035378999998, 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:28.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670021108, 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.750 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:28.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.754 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.670011791, 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:28.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:28.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:28.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:28.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:28.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:28.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:28.760 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.668342779, 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:28.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:28.765 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.668307716000001, 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:28.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:28.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.668282689000002, 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.770 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:28.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:28.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.668258381000001, 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:28.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:28.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2675 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3817472~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:28.779 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.666066129999997, 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060019590999999997, 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2675 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031716+0000", 2024-09-15T00:10:28.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3200000000000005e-07 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031719+0000", 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8490000000000002e-06 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031726+0000", 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0849999999999999e-06 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031731+0000", 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1290000000000004e-06 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091552+0000", 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059820489999999997 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091640+0000", 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7719000000000005e-05 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091735+0000", 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5786999999999996e-05 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.666041997999997, 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:28.784 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:28.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.666012625999997, 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:28.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:28.793 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.001509198999999, 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:28.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:28.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:10:28.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:10:28.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:28.799 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:28.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:28.801 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:28.799+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:28.853 INFO:teuthology.orchestra.run.smithi027.stdout:3.c 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.b 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.3 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.1f 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.1b 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.1 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.18 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.17 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.6 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.15 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.12 2024-09-15T00:10:28.854 INFO:teuthology.orchestra.run.smithi027.stdout:3.10 2024-09-15T00:10:28.855 INFO:teuthology.orchestra.run.smithi027.stdout:2.5 2024-09-15T00:10:28.855 INFO:teuthology.orchestra.run.smithi027.stdout:2.4 2024-09-15T00:10:28.855 INFO:teuthology.orchestra.run.smithi027.stdout:1.0 2024-09-15T00:10:28.902 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:28.900+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:28.950 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:28.950 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:28.950 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:28.950 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:28.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:29.043 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:29.041+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:29.104 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:29.102+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:29.106 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:29.107 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:29.115 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:29.180 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:10:29.196 INFO:teuthology.orchestra.run.smithi092.stdout:5880 write 0xbe8e000 thru 0xbe9a285 (0xc286 bytes) 2024-09-15T00:10:29.204 INFO:teuthology.orchestra.run.smithi092.stdout:5881 punch from 0x5ccfa00 to 0x5cd3e00, (0x4400 bytes) 2024-09-15T00:10:29.205 INFO:teuthology.orchestra.run.smithi092.stdout:5882 punch from 0x593ce00 to 0x5944c00, (0x7e00 bytes) 2024-09-15T00:10:29.227 INFO:teuthology.orchestra.run.smithi092.stdout:5884 write 0xd0ada00 thru 0xd0b3b4d (0x614e bytes) 2024-09-15T00:10:29.235 INFO:teuthology.orchestra.run.smithi092.stdout:5885 write 0xb70e600 thru 0xb713b68 (0x5569 bytes) 2024-09-15T00:10:29.235 INFO:teuthology.orchestra.run.smithi092.stdout:5888 write 0x8e7d000 thru 0x8e8156f (0x4570 bytes) 2024-09-15T00:10:29.235 INFO:teuthology.orchestra.run.smithi092.stdout:5890 write 0x6c39c00 thru 0x6c4305a (0x945b bytes) 2024-09-15T00:10:29.235 INFO:teuthology.orchestra.run.smithi092.stdout:5893 read 0x922aa00 thru 0x9231bbc (0x71bd bytes) 2024-09-15T00:10:29.236 INFO:teuthology.orchestra.run.smithi092.stdout:5897 write 0xb5bde00 thru 0xb5cc2fa (0xe4fb bytes) 2024-09-15T00:10:29.236 INFO:teuthology.orchestra.run.smithi092.stdout:5898 read 0x4258a00 thru 0x4268645 (0xfc46 bytes) 2024-09-15T00:10:29.236 INFO:teuthology.orchestra.run.smithi092.stdout:5901 read 0x801e800 thru 0x8025caa (0x74ab bytes) 2024-09-15T00:10:29.236 INFO:teuthology.orchestra.run.smithi092.stdout:5903 read 0x583000 thru 0x58bd63 (0x8d64 bytes) 2024-09-15T00:10:29.236 INFO:teuthology.orchestra.run.smithi092.stdout:5904 read 0x48bb000 thru 0x48c9be6 (0xebe7 bytes) 2024-09-15T00:10:29.242 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:29.240+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:29.242 INFO:teuthology.orchestra.run.smithi092.stdout:5905 trunc from 0xd0b3c00 to 0x9ffbe00 2024-09-15T00:10:29.276 INFO:teuthology.orchestra.run.smithi092.stdout:5906 trunc from 0x9ffbe00 to 0xc551800 2024-09-15T00:10:29.277 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:29.277 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.249574725000002, 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.249552807000001, 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.249531535000003, 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.287 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:29.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:29.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.291 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.247790275, 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:29.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:29.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.247768368999999, 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.296 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:29.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.300 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.247746276999997, 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:29.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:29.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:29.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:29.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:29.305 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.247714537999997, 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:29.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.247416334999997, 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:29.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.247382629999997, 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:29.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.243080556999999, 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:29.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:29.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.242755686999999, 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.324 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:29.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.242345282000002, 2024-09-15T00:10:29.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:29.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.242290379000003, 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:29.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.337 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.242258344, 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:29.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:29.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.240780514000001, 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.343 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.343 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.343 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:29.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.240758798999998, 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.348 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.240737320000001, 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.238724216000001, 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.238704575, 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 59.238658027, 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:29.371 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:29.372 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:29.369+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:29.373 INFO:teuthology.orchestra.run.smithi092.stdout:5907 write 0x649c00 thru 0x659189 (0xf58a bytes) 2024-09-15T00:10:29.373 INFO:teuthology.orchestra.run.smithi092.stdout:5908 read 0x6111e00 thru 0x611685d (0x4a5e bytes) 2024-09-15T00:10:29.373 INFO:teuthology.orchestra.run.smithi092.stdout:5909 read 0x4595800 thru 0x459b115 (0x5916 bytes) 2024-09-15T00:10:29.373 INFO:teuthology.orchestra.run.smithi092.stdout:5912 trunc from 0xc551800 to 0x1433400 2024-09-15T00:10:29.373 INFO:teuthology.orchestra.run.smithi092.stdout:5913 trunc from 0x1433400 to 0xbb72c00 2024-09-15T00:10:29.373 INFO:teuthology.orchestra.run.smithi092.stdout:5914 clone 40 order 22 su 32768 sc 7 2024-09-15T00:10:29.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:29.374 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 3.6 --file /var/log/ceph/exp.3.6.0 2024-09-15T00:10:29.473 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:29.471+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:29.477 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:29.477 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:29.477 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:29.477 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:29.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:29.574 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:29.572+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:29.588 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:29.588 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:29.588 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:29.588 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:29.589 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:29.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:29.675 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:29.672+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.785409416, 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.700 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:29.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:29.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.785389110000001, 2024-09-15T00:10:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:29.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:29.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.785367170000001, 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.783509399, 2024-09-15T00:10:29.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:29.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.783479720000001, 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:29.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:29.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.783456627, 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.724 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:29.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.783431454, 2024-09-15T00:10:29.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:29.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.732 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.783410813, 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:29.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.783390272, 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:29.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:29.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:29.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.783366790000001, 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781503133999999, 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781473365, 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781451186, 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781423935999999, 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781397342, 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781378009000001, 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.779329600000001, 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.779314367, 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.014037847999999, 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:10:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.967947177999999, 2024-09-15T00:10:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:10:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:10:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:10:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:29.797 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:29.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:29.799 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:29.796+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:29.898 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:29.896+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:29.943 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:29.943 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:29.943 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:29.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:29.999 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:29.997+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:30.095 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:30.095 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:30.095 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:30.095 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:30.095 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:30.101 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:30.099+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:30.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:10:30.193 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:30.201 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:30.199+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:30.213 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:30.213 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:30.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:30.213 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:30.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.213 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.213 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.328036855999997, 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:30.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:30.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.218 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.327998226000005, 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:30.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:30.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:30.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:30.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:30.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:30.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:30.223 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.327972289000002, 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:30.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:30.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:30.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:30.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:30.228 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.327956220999994, 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.327815126000004, 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:30.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.326958340000004, 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:30.237 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:30.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.326923324000006, 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.326897720999995, 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:30.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:30.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.251 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.326723455000007, 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.252 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.326492316, 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:30.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:30.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.260 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.325680371999994, 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:30.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:30.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.325600632000004, 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:30.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:30.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:30.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:30.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:30.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.272 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.325395607000004, 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.325184074000006, 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:30.277 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:30.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.322628066999997, 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:30.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:30.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:30.286 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.320644960999999, 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:30.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:30.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.320625390000004, 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:30.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.295 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.049882539999999, 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:30.296 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.144131045, 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.300 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:30.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.527043604999999, 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:30.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:30.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:30.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:30.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:30.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:30.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:30.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:30.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:30.310 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:30.311 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone39 from 0xbb72c00 (overlap 0x41a800) to 0x20d400 2024-09-15T00:10:30.312 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:30.309+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:30.312 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:30.332 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:30.363 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:30.363 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:30.384 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:30.402 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:30.405 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:30.405 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:30.413 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:30.411+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:30.426 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:30.426 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:30.437 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:30.522 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:30.521+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:30.547 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-15T00:10:30.587 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:30.587 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:30.595 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:30.615 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:30.612+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:30.716 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:30.713+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:30.752 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:30.752 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:30.761 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:30.817 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:30.816+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:30.879 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:30.880 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:30.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:30.918 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:30.916+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:31.019 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:31.017+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:31.038 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:31.039 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:31.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:31.120 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:31.118+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:31.153 INFO:teuthology.orchestra.run.smithi092.stdout:5915 read 0x9eb2600 thru 0x9eb563e (0x303f bytes) 2024-09-15T00:10:31.154 INFO:teuthology.orchestra.run.smithi092.stdout:5916 punch from 0x2dfae00 to 0x2e05c00, (0xae00 bytes) 2024-09-15T00:10:31.155 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:31.155 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:31.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:31.174 INFO:teuthology.orchestra.run.smithi092.stdout:5920 write 0xd315600 thru 0xd3212c4 (0xbcc5 bytes) 2024-09-15T00:10:31.176 INFO:teuthology.orchestra.run.smithi092.stdout:5922 punch from 0xa6f4a00 to 0xa6f8600, (0x3c00 bytes) 2024-09-15T00:10:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:5923 read 0x6714200 thru 0x671f279 (0xb07a bytes) 2024-09-15T00:10:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:5926 read 0x3ed1400 thru 0x3edc5cf (0xb1d0 bytes) 2024-09-15T00:10:31.180 INFO:teuthology.orchestra.run.smithi092.stdout:5927 write 0x1293c00 thru 0x12a0238 (0xc639 bytes) 2024-09-15T00:10:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:5929 read 0x3a8d000 thru 0x3a954cc (0x84cd bytes) 2024-09-15T00:10:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:5930 write 0x675b400 thru 0x676af99 (0xfb9a bytes) 2024-09-15T00:10:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:5931 read 0xc1ebc00 thru 0xc1f6850 (0xac51 bytes) 2024-09-15T00:10:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:5933 read 0x770ec00 thru 0x7718cfd (0xa0fe bytes) 2024-09-15T00:10:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:5938 write 0xcdb8600 thru 0xcdbf4a2 (0x6ea3 bytes) 2024-09-15T00:10:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:5939 read 0xa0e8400 thru 0xa0e9020 (0xc21 bytes) 2024-09-15T00:10:31.181 INFO:teuthology.orchestra.run.smithi092.stdout:5941 write 0x7caf400 thru 0x7cbe2c1 (0xeec2 bytes) 2024-09-15T00:10:31.187 INFO:teuthology.orchestra.run.smithi092.stdout:5942 trunc from 0xd321400 to 0x7814c00 2024-09-15T00:10:31.220 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:31.219+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:31.239 INFO:teuthology.orchestra.run.smithi092.stdout:5943 trunc from 0x7814c00 to 0xe103000 2024-09-15T00:10:31.249 INFO:teuthology.orchestra.run.smithi092.stdout:5944 read 0x2dd7c00 thru 0x2dddca5 (0x60a6 bytes) 2024-09-15T00:10:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:5945 read 0x68b5800 thru 0x68bd093 (0x7894 bytes) 2024-09-15T00:10:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:5948 read 0x9427e00 thru 0x9437d3f (0xff40 bytes) 2024-09-15T00:10:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:5951 write 0x23c1400 thru 0x23c40d5 (0x2cd6 bytes) 2024-09-15T00:10:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:5955 read 0x2e89e00 thru 0x2e92ec9 (0x90ca bytes) 2024-09-15T00:10:31.251 INFO:teuthology.orchestra.run.smithi092.stdout:5956 clone 41 order 21 su 16384 sc 13 2024-09-15T00:10:31.279 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:31.279 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:31.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:31.311 INFO:teuthology.orchestra.run.smithi027.stderr:Exporting 3.6 info 3.6( v 55'115 (0'0,55'115] local-lis/les=20/21 n=23 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-15T00:10:31.318 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:6153c2de:::rbd_data.11291d0c7d7c.0000000000000008:16# 2024-09-15T00:10:31.321 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:6153c2de:::rbd_data.11291d0c7d7c.0000000000000008:head# 2024-09-15T00:10:31.321 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:6168fac1:::rbd_data.11291d0c7d7c.0000000000000002:16# 2024-09-15T00:10:31.331 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:31.330+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:31.332 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:6168fac1:::rbd_data.11291d0c7d7c.0000000000000002:head# 2024-09-15T00:10:31.339 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:619a9c01:::rbd_data.112981949474.00000000000000b3:head# 2024-09-15T00:10:31.340 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:61d17d30:::rbd_data.1129aa143e51.000000000000017e:5# 2024-09-15T00:10:31.340 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:61d17d30:::rbd_data.1129aa143e51.000000000000017e:head# 2024-09-15T00:10:31.340 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:635f65a8:::rbd_data.112981949474.000000000000019f:head# 2024-09-15T00:10:31.341 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:63c09de1:::rbd_data.1129feb48f4b.000000000000005a:4# 2024-09-15T00:10:31.343 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:63c09de1:::rbd_data.1129feb48f4b.000000000000005a:head# 2024-09-15T00:10:31.343 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:640b0afe:::rbd_data.112981949474.0000000000000169:head# 2024-09-15T00:10:31.343 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:642fe358:::rbd_data.1129b452887f.0000000000000030:head# 2024-09-15T00:10:31.343 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:6471fc50:::rbd_data.112981949474.0000000000000014:head# 2024-09-15T00:10:31.344 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:64ef3dea:::rbd_data.11299af7e2c0.0000000000000007:15# 2024-09-15T00:10:31.346 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:64ef3dea:::rbd_data.11299af7e2c0.0000000000000007:head# 2024-09-15T00:10:31.346 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:654484c1:::rbd_data.11291357fc5b.0000000000000004:head# 2024-09-15T00:10:31.348 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:655148c9:::rbd_data.1129aa143e51.0000000000000010:head# 2024-09-15T00:10:31.348 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:65bea106:::rbd_data.112981949474.0000000000000018:head# 2024-09-15T00:10:31.349 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:66099f75:::rbd_data.1129aa143e51.0000000000000068:head# 2024-09-15T00:10:31.349 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:6681a010:::rbd_data.11299af7e2c0.0000000000000012:15# 2024-09-15T00:10:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:6681a010:::rbd_data.11299af7e2c0.0000000000000012:head# 2024-09-15T00:10:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:66f536f5:::rbd_data.112951ba645b.0000000000000038:13# 2024-09-15T00:10:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:Read #3:66f536f5:::rbd_data.112951ba645b.0000000000000038:head# 2024-09-15T00:10:31.352 INFO:teuthology.orchestra.run.smithi027.stderr:Export successful 2024-09-15T00:10:31.389 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:31.389 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:31.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:31.422 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:31.421+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:31.523 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:31.522+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:31.540 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:31.540 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:31.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:31.632 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:31.631+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:31.692 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:31.693 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:31.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:31.725 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:31.724+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:31.826 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:31.825+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:31.844 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:31.844 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:31.852 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:31.868 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 3.6 2024-09-15T00:10:31.953 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:31.952+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:31.995 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:31.995 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:32.003 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:32.048 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:32.047+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:32.114 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:32.114 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:32.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:32.163 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:32.162+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:32.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:10:32.229 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:32.228+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:32.278 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:32.278 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:32.281 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone40 2024-09-15T00:10:32.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:32.330 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:32.329+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:32.400 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:32.400 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:32.407 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:32.410 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:32.451 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:32.450+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:32.523 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:32.532 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:32.531+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:32.540 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:32.541 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:32.558 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:32.565 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:32.566 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:32.575 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:32.576 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:32.577 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:32.594 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:32.594 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:32.597 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:32.597 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:32.639 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:32.638+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:32.685 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:32.685 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:32.685 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:32.685 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:32.685 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:32.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:32.695 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-15T00:10:32.734 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:32.733+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:32.782 INFO:teuthology.orchestra.run.smithi092.stdout:5958 write 0xe851200 thru 0xe851631 (0x432 bytes) 2024-09-15T00:10:32.783 INFO:teuthology.orchestra.run.smithi092.stdout:5959 read 0x2e4a200 thru 0x2e51693 (0x7494 bytes) 2024-09-15T00:10:32.783 INFO:teuthology.orchestra.run.smithi092.stdout:5960 punch from 0x1662400 to 0x166c000, (0x9c00 bytes) 2024-09-15T00:10:32.784 INFO:teuthology.orchestra.run.smithi092.stdout:5961 read 0xb45a000 thru 0xb4617f2 (0x77f3 bytes) 2024-09-15T00:10:32.784 INFO:teuthology.orchestra.run.smithi092.stdout:5964 punch from 0x8e6e000 to 0x8e78c00, (0xac00 bytes) 2024-09-15T00:10:32.822 INFO:teuthology.orchestra.run.smithi092.stdout:5966 write 0xed3f600 thru 0xed42270 (0x2c71 bytes) 2024-09-15T00:10:32.823 INFO:teuthology.orchestra.run.smithi092.stdout:5967 write 0x4b1e600 thru 0x4b21f28 (0x3929 bytes) 2024-09-15T00:10:32.823 INFO:teuthology.orchestra.run.smithi092.stdout:5968 write 0x3d95c00 thru 0x3d9be20 (0x6221 bytes) 2024-09-15T00:10:32.823 INFO:teuthology.orchestra.run.smithi092.stdout:5971 write 0x43cbe00 thru 0x43d0cf9 (0x4efa bytes) 2024-09-15T00:10:32.823 INFO:teuthology.orchestra.run.smithi092.stdout:5972 read 0x85b5400 thru 0x85b602a (0xc2b bytes) 2024-09-15T00:10:32.823 INFO:teuthology.orchestra.run.smithi092.stdout:5974 write 0xbe40600 thru 0xbe491cc (0x8bcd bytes) 2024-09-15T00:10:32.823 INFO:teuthology.orchestra.run.smithi092.stdout:5976 write 0x61d6000 thru 0x61d948d (0x348e bytes) 2024-09-15T00:10:32.834 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:32.833+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:32.835 INFO:teuthology.orchestra.run.smithi092.stdout:5979 trunc from 0xed42400 to 0x491fe00 2024-09-15T00:10:32.871 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:32.871 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:32.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:32.871 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:32.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.871 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.842139400000001, 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.872 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:32.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.842117481999999, 2024-09-15T00:10:32.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:32.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.842096210000001, 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:32.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:32.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:32.887 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.840354949999998, 2024-09-15T00:10:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.840333043999998, 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:32.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.840310952000003, 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.898 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:32.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.840279213000002, 2024-09-15T00:10:32.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:32.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:32.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.839981010000002, 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.908 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.908 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.908 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:32.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.908 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:32.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.839947305000003, 2024-09-15T00:10:32.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:32.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:32.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.835645231999997, 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.917 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:32.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.921 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.835320361999997, 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:32.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.834909957000001, 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:32.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:32.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:32.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:32.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:32.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:32.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.834855054000002, 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:32.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:32.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.834823018999998, 2024-09-15T00:10:32.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:32.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.833345188999999, 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.833323473999997, 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:32.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.833301994999999, 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.831288891, 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:32.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.831269249999998, 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.962 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:32.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:32.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:32.966 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.831222701999998, 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:32.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:32.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:32.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:32.971 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.971 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.971 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.971 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:32.971 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:32.971 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:32.972 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:32.970+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:32.972 INFO:teuthology.orchestra.run.smithi092.stdout:5980 trunc from 0x491fe00 to 0x38aa00 2024-09-15T00:10:32.973 INFO:teuthology.orchestra.run.smithi092.stdout:5981 read 0x191400 thru 0x1a0dc2 (0xf9c3 bytes) 2024-09-15T00:10:32.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:32.993 INFO:teuthology.orchestra.run.smithi092.stdout:5982 write 0x88fa000 thru 0x88fc1c0 (0x21c1 bytes) 2024-09-15T00:10:32.993 INFO:teuthology.orchestra.run.smithi092.stdout:5983 read 0x61bc400 thru 0x61c1897 (0x5498 bytes) 2024-09-15T00:10:32.993 INFO:teuthology.orchestra.run.smithi092.stdout:5987 read 0x3aa5400 thru 0x3aac452 (0x7053 bytes) 2024-09-15T00:10:32.994 INFO:teuthology.orchestra.run.smithi092.stdout:5990 trunc from 0x88fc200 to 0x7150e00 2024-09-15T00:10:33.017 INFO:teuthology.orchestra.run.smithi092.stdout:5991 write 0xc73600 thru 0xc777ab (0x41ac bytes) 2024-09-15T00:10:33.018 INFO:teuthology.orchestra.run.smithi092.stdout:5992 read 0x5173000 thru 0x5176df2 (0x3df3 bytes) 2024-09-15T00:10:33.040 INFO:teuthology.orchestra.run.smithi092.stdout:5994 write 0xa032a00 thru 0xa03b43c (0x8a3d bytes) 2024-09-15T00:10:33.043 INFO:teuthology.orchestra.run.smithi092.stdout:5995 trunc from 0xa03b600 to 0x7e6dc00 2024-09-15T00:10:33.073 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:33.072+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:33.083 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:33.083 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:33.083 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:33.083 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:33.091 INFO:teuthology.orchestra.run.smithi092.stdout:5996 write 0xcaa1200 thru 0xcaab948 (0xa749 bytes) 2024-09-15T00:10:33.091 INFO:teuthology.orchestra.run.smithi092.stdout:6000 punch from 0xa342000 to 0xa34f800, (0xd800 bytes) 2024-09-15T00:10:33.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:33.174 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:33.173+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:33.195 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:33.195 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:33.195 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:33.195 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:33.195 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:33.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:33.205 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:10:33.242 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-15T00:10:33.275 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:33.274+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:33.320 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:33.320 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:33.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:33.320 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:33.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.405121751999999, 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:33.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.405101446, 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:33.325 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:33.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.405079506, 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:33.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:33.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.403221734999999, 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.335 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:33.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:33.339 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.403192056000002, 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:33.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:33.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.403168962999999, 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.344 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:33.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:33.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.403143790000001, 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:33.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:33.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.403123148999999, 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:33.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:33.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:33.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.403102608000001, 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:33.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:33.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.403079126000002, 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.40121547, 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.401185700999999, 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.401163522000001, 2024-09-15T00:10:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.401136271999999, 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.401109678000001, 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.401090345, 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.396 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.399041936, 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:33.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.399026703000001, 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.401 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:33.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.633750184, 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:33.406 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:10:33.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.587659514, 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.411 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:10:33.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.416 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:33.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:33.418 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:33.416+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:33.519 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:33.517+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:33.597 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:33.598 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:33.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:33.651 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:33.650+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:33.721 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:33.719+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:33.744 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:33.745 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:33.755 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:33.820 INFO:teuthology.orchestra.run.smithi027.stdout: marking collection for removal 2024-09-15T00:10:33.820 INFO:teuthology.orchestra.run.smithi027.stdout:setting '_remove' omap key 2024-09-15T00:10:33.820 INFO:teuthology.orchestra.run.smithi027.stdout:finish_remove_pgs 3.6_head removing 3.6 2024-09-15T00:10:33.832 INFO:teuthology.orchestra.run.smithi027.stdout:Remove successful 2024-09-15T00:10:33.840 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:33.839+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:33.901 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:33.901 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:33.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:33.901 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.016212327000005, 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.016173696999999, 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.016147759999996, 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.016131692000002, 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.015990596999998, 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.015133810999998, 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.015098795, 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.015073192000003, 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:33.941 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.014898926000001, 2024-09-15T00:10:33.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:33.941 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.941 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.944 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.944 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.014667786999993, 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.013855843000002, 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.013776102999998, 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.013571077999998, 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.013359545, 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.010803538000005, 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.008820431999993, 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.008800860999997, 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.738058011, 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.832306515999999, 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.215219076, 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:34.003 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:34.005 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:34.003+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:34.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:34.105 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:34.104+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:34.120 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:34.120 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:34.120 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:34.120 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:34.129 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:34.213 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:34.212+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:34.280 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:34.280 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:34.280 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:34.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:34.331 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:34.330+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:34.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-15T00:10:34.408 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:34.406+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:34.440 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:34.440 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:34.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:34.441 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:34.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.441 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.441 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:34.441 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.410059484000001, 2024-09-15T00:10:34.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:34.441 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:34.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.442 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:34.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:34.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:34.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:34.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:34.447 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.410039737999995, 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:34.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.410019788, 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:34.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:34.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.457 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.409992287999998, 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:34.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.409975446999994, 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:34.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.409959448999999, 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:34.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.471 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.409936461000001, 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:34.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.409910577000005, 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:34.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.480 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.40989415, 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.409879879000002, 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.409870561999995, 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.408201550000001, 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:34.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.408166487000003, 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.504 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.408141459999996, 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:34.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.408117152000003, 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:34.509 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:34.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.405900768999999, 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:34.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.518 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.405871396999999, 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.519 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:34.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.523 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.741367970000001, 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:10:34.524 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:34.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:10:34.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:34.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5644.0:745 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.2657740950000003, 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227015319999999, 2024-09-15T00:10:34.530 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5644", 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1022470035", 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 745 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171868+0000", 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-06 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171876+0000", 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.593e-06 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171890+0000", 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4347e-05 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171897+0000", 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8449999999999997e-06 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171949+0000", 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2206000000000001e-05 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171959+0000", 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7829999999999993e-06 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194277+0000", 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0223181750000001 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194432+0000", 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155329 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194568+0000", 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013594599999999999 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.535 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20982 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.2657064049999995, 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0231241369999999, 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.536 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20982 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171935+0000", 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1200000000000003e-07 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171938+0000", 2024-09-15T00:10:34.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9550000000000001e-06 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171944+0000", 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171948+0000", 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8519999999999997e-06 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171977+0000", 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9539999999999998e-05 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171981+0000", 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.478e-06 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194852+0000", 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0228714160000001 2024-09-15T00:10:34.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194908+0000", 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5837000000000001e-05 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.195059+0000", 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015047699999999999 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.541 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.542 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:34.543 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:34.543 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:34.542+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:34.599 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:34.643 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:34.642+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:34.691 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:34.691 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:34.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:34.733 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:34.744 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:34.744+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:34.808 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:34.808 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:34.809 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:34.809 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:34.809 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:34.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:34.845 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:34.843+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:34.946 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:34.945+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:34.952 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:34.952 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:34.966 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:34.966 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.938639417999994, 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.967 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:34.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:34.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.938617500000007, 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:34.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:34.976 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.938596227999994, 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.977 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:34.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.936854968000006, 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.982 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:34.986 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.936833062000005, 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.987 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:34.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:34.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.991 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.936810969999996, 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.992 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.936779231000003, 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:34.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:34.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.001 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.936481028000003, 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:35.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.936447322999996, 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:35.006 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:35.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.010 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.932145250000005, 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:35.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.931820380000005, 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:35.015 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:35.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:35.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.931409974999994, 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.020 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:35.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.024 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.931355072000002, 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.025 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.029 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.931323036999999, 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:35.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:35.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:35.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:35.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:35.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.929845207, 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:35.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.039 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.929823491999997, 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.040 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:35.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:35.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.044 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.929802013, 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.927788909, 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.050 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:35.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.927769268000006, 2024-09-15T00:10:35.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:35.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.058 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 64.927722720000006, 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:35.059 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:35.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:35.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:35.063 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:35.064 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:35.064 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:35.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:35.066 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:35.066 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:35.067 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:35.067 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:35.067 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:35.068 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:35.068 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:35.063+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:35.165 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:35.163+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:35.170 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:35.170 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:35.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:35.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:10:35.266 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:35.265+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:35.338 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:35.338 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:35.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:35.392 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:35.391+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:35.461 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:35.461 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:35.469 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:35.468+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:35.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:35.570 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:35.569+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:35.612 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:35.612 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:35.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:35.671 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:35.670+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:35.724 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:35.724 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:35.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:35.772 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:35.771+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:35.835 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:35.835 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:35.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:35.889 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:35.888+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:35.954 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:35.954 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:35.963 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:36.004 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:36.003+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:36.075 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:36.074+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:36.110 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:36.110 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:36.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:36.203 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:36.202+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:36.214 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:10:36.271 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:36.271 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:36.276 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:36.275+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:36.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:36.365 INFO:teuthology.orchestra.run.smithi079.stdout:3.e 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.9 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.b 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.1c 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.1a 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.1 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.3 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.1e 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.19 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.a 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.1b 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.14 2024-09-15T00:10:36.366 INFO:teuthology.orchestra.run.smithi079.stdout:3.11 2024-09-15T00:10:36.367 INFO:teuthology.orchestra.run.smithi079.stdout:3.6 2024-09-15T00:10:36.367 INFO:teuthology.orchestra.run.smithi079.stdout:3.15 2024-09-15T00:10:36.367 INFO:teuthology.orchestra.run.smithi079.stdout:2.7 2024-09-15T00:10:36.367 INFO:teuthology.orchestra.run.smithi079.stdout:2.5 2024-09-15T00:10:36.367 INFO:teuthology.orchestra.run.smithi079.stdout:2.4 2024-09-15T00:10:36.377 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:36.375+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:36.427 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:36.428 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:36.436 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:36.478 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:36.477+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:36.542 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:36.542 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:36.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:36.594 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:36.593+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:36.657 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:36.658 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:36.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:36.707 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:36.706+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:36.770 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:36.770 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:36.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:36.780 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:36.778+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:36.880 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:36.879+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:36.884 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:36.885 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:36.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.6.0 2024-09-15T00:10:36.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:36.981 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:36.979+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:37.036 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:37.036 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:37.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:37.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:37.112 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:37.111+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:37.174 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:37.183 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:37.182+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:37.186 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:37.186 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:37.191 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:37.191 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:37.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:37.208 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:37.225 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:37.226 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:37.231 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:37.250 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:37.250 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:37.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:37.254 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:37.295 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:37.294+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:37.344 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:37.344 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:37.345 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:37.345 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:37.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:37.397 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:37.396+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:37.457 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:37.457 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.572403507000004, 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:37.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.462 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.572364876999998, 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.463 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:37.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:37.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.572338939999995, 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:37.467 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:37.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.471 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.572322872000001, 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.472 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:37.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.572181776999997, 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.477 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:37.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:37.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.571324990999997, 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:37.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.571289974999999, 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.487 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:37.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:37.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:37.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:37.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:37.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.571264372000002, 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:37.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:37.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:37.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.571090106, 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:37.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:37.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.570858967000007, 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.501 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:37.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:37.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.570047023000001, 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.507 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:37.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.507 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:37.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:37.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.569967282999997, 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:37.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:37.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.569762257999997, 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:37.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:37.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:37.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:37.520 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.569550724999999, 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:37.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.566994718000004, 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:37.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:37.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:37.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:37.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.565011612000006, 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:37.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.564992040999996, 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:37.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:37.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.539 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 35.294249190999999, 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:37.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:37.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:37.544 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.388497696000002, 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:37.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:37.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:37.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:37.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:37.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:37.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:37.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:37.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:37.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.771410255999999, 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:37.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:37.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:37.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:37.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:37.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:37.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:37.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:37.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:37.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:37.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:37.556 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:37.557 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:37.558 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:37.556+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:37.658 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:37.657+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:37.665 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:37.665 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:37.665 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:37.665 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:37.673 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:37.758 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:37.757+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:37.824 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:37.825 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:37.825 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:37.825 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:37.825 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:37.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:37.861 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:37.860+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:37.963 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:37.961+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:37.984 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:37.984 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:37.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954647527999995, 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:37.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:37.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954627782000003, 2024-09-15T00:10:37.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:37.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:37.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954607831999994, 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:37.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:37.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:37.998 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954580332000006, 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954563491000002, 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.003 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:38.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:38.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954547493000007, 2024-09-15T00:10:38.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:38.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:38.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954524504999995, 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.012 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954498620999999, 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:38.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954482193999993, 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:38.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:38.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954467922999996, 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.025 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:38.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.954458606000003, 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.030 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.952789593999995, 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:38.034 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:38.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.038 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.952754530999997, 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:38.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.043 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.952729504000004, 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.044 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:38.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.952705195999997, 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:38.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.950488813000007, 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:38.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.057 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.950459441000007, 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:38.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:38.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.285956014, 2024-09-15T00:10:38.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:38.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:10:38.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:10:38.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5644.0:745 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:38.068 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.8103621390000004, 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227015319999999, 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5644", 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1022470035", 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 745 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:38.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171868+0000", 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-06 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171876+0000", 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.593e-06 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171890+0000", 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4347e-05 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171897+0000", 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8449999999999997e-06 2024-09-15T00:10:38.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171949+0000", 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2206000000000001e-05 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171959+0000", 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7829999999999993e-06 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194277+0000", 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0223181750000001 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194432+0000", 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155329 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194568+0000", 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013594599999999999 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20982 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.8102944490000006, 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0231241369999999, 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20982 2024-09-15T00:10:38.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171935+0000", 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1200000000000003e-07 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171938+0000", 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9550000000000001e-06 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171944+0000", 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171948+0000", 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8519999999999997e-06 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171977+0000", 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9539999999999998e-05 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171981+0000", 2024-09-15T00:10:38.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.478e-06 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194852+0000", 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0228714160000001 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194908+0000", 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5837000000000001e-05 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.195059+0000", 2024-09-15T00:10:38.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015047699999999999 2024-09-15T00:10:38.079 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.079 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.079 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.079 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.079 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.079 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:38.080 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:38.081 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:38.079+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:38.180 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:38.179+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:38.231 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:38.231 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:38.231 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:38.231 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:38.231 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:10:38.244 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:38.281 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:38.279+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:38.381 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:38.380+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:38.412 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:38.412 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:38.412 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:38.413 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:38.413 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:38.422 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:38.483 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:38.481+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:38.532 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.532 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.503083551000003, 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:38.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:38.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:38.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:38.537 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.503061633000002, 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:38.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:38.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.543 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.503040361000004, 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.544 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:38.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.548 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.501299101000001, 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.549 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.550 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:38.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:38.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:38.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.501277195, 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:38.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:38.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.501255103000005, 2024-09-15T00:10:38.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:38.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.501223363999998, 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.500925160999998, 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.500891456000005, 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.580 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.581 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:38.582 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.496589383, 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.586 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:38.587 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.588 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.496264513, 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:38.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:38.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:38.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:38.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.495854108000003, 2024-09-15T00:10:38.594 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:38.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.495799204999997, 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.495767169999993, 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:38.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:38.610 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.494289339999995, 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:38.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.615 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.494267625000006, 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:38.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.621 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.494246145999995, 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:38.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.492233041999995, 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:38.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:38.632 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.492213401000001, 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:38.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.492166853000001, 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:38.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:38.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:38.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:38.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:38.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:38.643 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.643 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.643 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.643 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:38.643 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:38.643 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:38.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:get_pg_num_history pg_num_history pg_num_history(e55 pg_nums {1={12=1},2={15=8},3={20=32}} deleted_pools ) 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:Importing pgid 3.6 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:6153c2de:::rbd_data.11291d0c7d7c.0000000000000008:16# 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:6153c2de:::rbd_data.11291d0c7d7c.0000000000000008:head# 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 16=[]:{16=[16]} 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:6168fac1:::rbd_data.11291d0c7d7c.0000000000000002:16# 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:6168fac1:::rbd_data.11291d0c7d7c.0000000000000002:head# 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 16=[]:{16=[16]} 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:619a9c01:::rbd_data.112981949474.00000000000000b3:head# 2024-09-15T00:10:38.649 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:61d17d30:::rbd_data.1129aa143e51.000000000000017e:5# 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:61d17d30:::rbd_data.1129aa143e51.000000000000017e:head# 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 5=[]:{5=[5]} 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:635f65a8:::rbd_data.112981949474.000000000000019f:head# 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:63c09de1:::rbd_data.1129feb48f4b.000000000000005a:4# 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:63c09de1:::rbd_data.1129feb48f4b.000000000000005a:head# 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 4=[]:{4=[4]} 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:640b0afe:::rbd_data.112981949474.0000000000000169:head# 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:642fe358:::rbd_data.1129b452887f.0000000000000030:head# 2024-09-15T00:10:38.650 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:10:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:6471fc50:::rbd_data.112981949474.0000000000000014:head# 2024-09-15T00:10:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:10:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:64ef3dea:::rbd_data.11299af7e2c0.0000000000000007:15# 2024-09-15T00:10:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:64ef3dea:::rbd_data.11299af7e2c0.0000000000000007:head# 2024-09-15T00:10:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 15=[]:{15=[15]} 2024-09-15T00:10:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:654484c1:::rbd_data.11291357fc5b.0000000000000004:head# 2024-09-15T00:10:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:10:38.651 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:655148c9:::rbd_data.1129aa143e51.0000000000000010:head# 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:65bea106:::rbd_data.112981949474.0000000000000018:head# 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 0=[]:{} 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:66099f75:::rbd_data.1129aa143e51.0000000000000068:head# 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 5=[]:{} 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:6681a010:::rbd_data.11299af7e2c0.0000000000000012:15# 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:6681a010:::rbd_data.11299af7e2c0.0000000000000012:head# 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 15=[]:{15=[15]} 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:66f536f5:::rbd_data.112951ba645b.0000000000000038:13# 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:Write #3:66f536f5:::rbd_data.112951ba645b.0000000000000038:head# 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:snapset 13=[]:{13=[13]} 2024-09-15T00:10:38.652 INFO:teuthology.orchestra.run.smithi027.stdout:write_pg epoch 55 info 3.6( v 55'115 (0'0,55'115] local-lis/les=20/21 n=23 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-15T00:10:38.653 INFO:teuthology.orchestra.run.smithi027.stdout:Import successful 2024-09-15T00:10:38.653 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:38.643+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:38.745 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:38.744+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:38.757 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:38.757 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:38.757 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:38.757 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:38.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:38.847 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:38.845+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:38.868 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:38.868 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:38.868 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:38.868 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:38.869 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:38.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:38.961 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:38.960+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:39.022 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:39.022 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:39.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:39.022 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.107818332000001, 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:39.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.107798026000001, 2024-09-15T00:10:39.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:39.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:39.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.107776086000001, 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:39.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.105918315, 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.037 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:39.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.105888636, 2024-09-15T00:10:39.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.105865543, 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:39.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:39.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:39.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:39.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:39.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:39.050 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.051 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.051 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.051 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.051 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:39.051 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.105840369999999, 2024-09-15T00:10:39.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:39.051 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:39.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:39.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:39.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.056 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.105819729, 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:39.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.105799187999999, 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:39.061 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:39.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:39.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.105775705999999, 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.066 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:39.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.103912050000002, 2024-09-15T00:10:39.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:39.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.103882281000001, 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:39.076 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:39.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.103860101999999, 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.103832852, 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.086 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:39.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:39.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.103806257999999, 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:39.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:39.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:39.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:39.095 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.103786925000001, 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:39.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:39.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.101738516000001, 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:39.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:39.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.101723282999998, 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:39.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:39.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:39.109 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.336446764000002, 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:10:39.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.290356094, 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:39.115 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:39.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:10:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:39.120 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:39.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /var/log/ceph/exp.3.6.0 2024-09-15T00:10:39.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:39.123 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:39.120+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:39.234 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:39.233+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:39.234 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:10:39.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:39.271 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:39.271 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:39.271 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:39.271 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:39.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:39.336 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:39.335+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:39.387 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:39.387 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:39.387 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:39.388 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:39.388 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:39.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:39.420 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:39.420 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:39.424 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:39.423+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:39.437 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:39.454 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:39.472 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:39.472 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:39.474 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:39.475 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:39.500 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:39.500 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:39.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:39.524 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:39.523+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:39.623 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:39.623 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:39.625 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:39.624+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:39.631 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:39.726 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:39.724+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:39.778 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:39.778 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:39.788 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:39.831 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:39.830+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:39.894 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:39.895 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:39.903 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:39.927 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:39.926+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:40.028 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:40.027+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:40.050 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:40.050 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:40.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:40.129 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:40.128+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:40.165 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:40.165 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:40.174 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:40.230 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:40.229+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:40.284 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:40.285 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:40.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:40.336 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:40.335+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:40.404 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:40.404 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:40.413 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:40.454 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:40.452+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:40.533 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:40.531+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:40.556 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:40.556 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:40.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:40.633 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:40.632+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:40.707 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:40.707 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:40.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:40.760 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:40.759+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:40.818 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:40.818 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:40.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:40.835 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:40.834+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:40.936 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:40.935+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:40.968 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:40.968 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:40.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:41.037 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:41.036+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:41.079 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:41.080 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:41.088 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:41.138 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:41.136+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:41.209 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:41.209 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:41.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:41.253 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:10:41.261 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:41.260+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:41.335 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:41.336 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:41.340 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:41.338+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:41.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:41.441 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:41.439+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:41.455 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:41.455 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:41.464 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:41.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:41.541 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:41.540+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:41.572 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:41.572 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:41.581 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:41.614 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:41.631 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:41.632 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:41.642 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:41.640+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:41.650 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:41.650 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:41.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:41.684 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:41.685 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:41.688 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:41.688 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:41.691 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:41.691 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:41.691 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:41.691 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:41.692 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:41.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:41.743 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:41.741+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:41.843 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:41.842+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:41.849 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.821299975000002, 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.821278057000001, 2024-09-15T00:10:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.821256785000003, 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.819515525, 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.819493618999999, 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.819471527000005, 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.819439787999997, 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.819141584999997, 2024-09-15T00:10:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.819107880000004, 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.814805806999999, 2024-09-15T00:10:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.814480936999999, 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.814070532000002, 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.814015628999996, 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.813983594000007, 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.812505763999994, 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.812484049000005, 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.812462569999994, 2024-09-15T00:10:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.810449465999994, 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.810429825, 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 71.810383277, 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:41.947 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:41.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:41.949 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:41.947+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:42.049 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:42.048+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:42.052 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:42.052 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:42.052 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:42.052 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:42.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:42.150 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:42.148+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:42.162 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:42.163 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:42.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:42.254 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:10:42.256 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:42.255+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:42.325 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:42.325 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:42.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:42.325 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:42.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.325 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.410441232, 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:42.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:42.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.410420926, 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:42.330 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:42.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.410398986000001, 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.335 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:42.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:42.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.408541215, 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.340 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:42.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:42.344 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.408511535999999, 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:42.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.408488443, 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.408463269999999, 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:42.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:42.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.408442629, 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:42.358 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:42.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.408422088000002, 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:42.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:42.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:42.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.408398605999999, 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:42.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.406534950000001, 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:42.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.406505181, 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:42.377 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:42.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.381 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.406483002000002, 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.382 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:42.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.406455751999999, 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:42.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.406429158000002, 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:42.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.406409825000001, 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.396 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:42.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:42.400 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.404361416, 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:42.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:42.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.404346183000001, 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.405 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:42.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.639069664000001, 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:10:42.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:10:42.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.592978993999999, 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:10:42.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:10:42.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.420 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:42.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:42.422 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:42.420+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:42.522 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:42.521+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:42.564 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:42.564 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:42.565 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:42.565 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:42.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:42.622 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:42.621+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:42.675 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:42.675 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:42.675 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:42.675 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:42.676 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:42.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:42.725 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:42.723+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:42.824 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:42.822+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:42.827 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.827 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.942374865999994, 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.828 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:42.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.942336236000003, 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:42.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.942310298999999, 2024-09-15T00:10:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.837 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:42.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.942294231000005, 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:42.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:42.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:42.846 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.942153136000002, 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:42.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:42.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.941296350000002, 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:42.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.941261334000004, 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.856 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:42.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:42.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.941235731000006, 2024-09-15T00:10:42.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:42.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:42.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.941061465000004, 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.940830325999997, 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:42.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:42.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.940018382000005, 2024-09-15T00:10:42.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:42.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:42.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.939938642000001, 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.879 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.883 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.939733617000002, 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:42.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.939522084000004, 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.888 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:42.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.936966076999994, 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:42.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.934982970999997, 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:42.897 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:42.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.934963400000001, 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:42.902 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:42.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:42.906 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.664220550000003, 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:42.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:42.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.758469054999999, 2024-09-15T00:10:42.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:42.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:42.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.141381615, 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:42.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:42.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:42.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:42.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:42.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:42.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:42.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:42.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:42.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:42.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:42.921 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:42.924 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:42.964 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:42.962+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:43.026 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:43.025+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:43.035 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:43.035 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:43.035 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:43.044 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:43.128 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:43.127+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:43.192 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:43.192 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:43.193 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:43.193 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:43.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:43.242 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:43.241+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:43.329 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:43.327+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:43.355 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325755666999996, 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.356 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325735921000003, 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:43.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:43.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325715970999994, 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:43.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.369 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325688471000007, 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:43.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325671630000002, 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.374 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:43.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:43.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325655631999993, 2024-09-15T00:10:43.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:43.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:43.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325632643999995, 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:43.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:43.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:43.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325606759999999, 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.387 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:43.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:43.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325590332999994, 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:43.391 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:43.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.395 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325576061999996, 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.325566745000003, 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.323897732999995, 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.408 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.323862669999997, 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:43.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:43.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.323837643000005, 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.413 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:43.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.323813334999997, 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:43.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:43.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.321596951999993, 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:43.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:43.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.321567580000007, 2024-09-15T00:10:43.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:43.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:43.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.657064153, 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:10:43.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:10:43.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:10:43.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5644.0:745 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:43.436 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.181470278000001, 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227015319999999, 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5644", 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1022470035", 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 745 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171868+0000", 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-06 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171876+0000", 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.593e-06 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171890+0000", 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4347e-05 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171897+0000", 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8449999999999997e-06 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171949+0000", 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2206000000000001e-05 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171959+0000", 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7829999999999993e-06 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194277+0000", 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0223181750000001 2024-09-15T00:10:43.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194432+0000", 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155329 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194568+0000", 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013594599999999999 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.441 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20982 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.181402587999999, 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0231241369999999, 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20982 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171935+0000", 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1200000000000003e-07 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171938+0000", 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9550000000000001e-06 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171944+0000", 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171948+0000", 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8519999999999997e-06 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171977+0000", 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9539999999999998e-05 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171981+0000", 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.478e-06 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194852+0000", 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0228714160000001 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194908+0000", 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5837000000000001e-05 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.195059+0000", 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015047699999999999 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.446 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:43.448 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:43.448 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:43.446+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:43.549 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:43.547+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:43.556 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:43.556 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:43.556 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:43.556 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:43.565 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:43.650 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:43.649+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:43.672 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:43.672 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:43.672 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:43.672 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:43.672 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:43.681 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:43.689 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:43.765 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:43.764+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.804924478999993, 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:43.833 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:43.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.838 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.804902561000006, 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:43.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:43.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.843 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.804881288999994, 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.844 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:43.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.803140029000005, 2024-09-15T00:10:43.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:43.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:43.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:43.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:43.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:43.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:43.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:43.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:43.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:43.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.803118123000004, 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:43.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:43.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.862 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.803096030999995, 2024-09-15T00:10:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.863 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:43.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.803064292000002, 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:43.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:43.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.873 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.802766089000002, 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.874 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:43.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.878 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.802732383999995, 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:43.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.798430311000004, 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.889 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.798105441000004, 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.890 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:43.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.797695035999993, 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:43.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.797640133000002, 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:43.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:43.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:43.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:43.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:43.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:43.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:43.908 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.797608097999998, 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.909 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:43.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.796130267999999, 2024-09-15T00:10:43.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:43.915 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.915 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.915 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.915 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:43.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:43.920 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.920 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.920 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.920 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.920 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.796108552999996, 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:43.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:43.925 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.796087073999999, 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.926 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.794073969999999, 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:43.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.794054329000005, 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:43.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 73.794007781000005, 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:43.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:43.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:43.947 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:43.948 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:43.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:43.950 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:43.950 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:43.951 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:43.951 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:43.951 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:43.948+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:44.049 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:44.048+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:44.054 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:44.054 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:44.054 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:44.054 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:44.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:44.082 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:44.082 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:44.082 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:44.099 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:44.099 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:44.102 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:44.103 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:44.150 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:44.149+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:44.165 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:44.165 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:44.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:44.193 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [0, 4] live_osds: [1, 2, 3, 5] 2024-09-15T00:10:44.193 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:10:44.194 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T00:10:44.194 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T00:10:44.251 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:44.250+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:44.270 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:10:44.282 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:44.282 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:44.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:44.376 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:44.374+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:44.441 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:44.441 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:44.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:44.491 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:44.489+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:44.499 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:10:44.499 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:44.499 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 105, 2024-09-15T00:10:44.499 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "4f532d31-28c1-4236-a74e-6392960baf2b", 2024-09-15T00:10:44.499 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2024-09-15T00:08:32.007191+0000", 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2024-09-15T00:10:44.258707+0000", 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2024-09-15T00:10:28.163783+0000", 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2024-09-15T00:10:22.135739+0000", 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5804032, 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "nodeep-scrub", 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2024-09-15T00:10:44.500 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 3, 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 6, 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "squid", 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2024-09-15T00:10:44.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:42.896076+0000", 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:10:44.502 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:10:44.503 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "14", 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T00:10:44.504 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:10:44.505 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:10:44.506 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:10:44.507 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 6.059999942779541, 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 6.059999942779541, 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.33000001311302185, 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 2, 2024-09-15T00:10:44.508 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "rbd", 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:49.173060+0000", 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:10:44.509 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 8, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 8, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 8, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 8, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 8, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:10:44.510 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "18", 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 18, 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:10:44.511 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:10:44.512 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:10:44.513 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.2400000095367432, 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.2400000095367432, 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.67000001668930054, 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2024-09-15T00:10:44.514 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "pool_client.0", 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T00:08:58.879058+0000", 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T00:10:44.515 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T00:10:44.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T00:10:44.516 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T00:10:44.516 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T00:10:44.516 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T00:10:44.516 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T00:10:44.516 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T00:10:44.516 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T00:10:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "105", 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 43, 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 105, 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T00:10:44.518 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T00:10:44.519 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T00:10:44.520 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.8700000047683716, 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.8700000047683716, 2024-09-15T00:10:44.521 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.67000001668930054, 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.25, 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.25, 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:10:44.522 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "56600dc2-b95d-47a7-b2b1-5ddc54c08341", 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 0, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 0, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 0, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 20, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 56, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.524 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:10:44.525 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:10:44.526 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2024-09-15T00:10:44.527 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1440638768 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/1440638768", 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/1440638768", 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6823/1440638768", 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6821/1440638768", 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: "exists" 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.528 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "ea8f0155-8472-4e87-89fd-7716d05065d0", 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 90, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:10:44.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.532 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3601510193 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/3601510193", 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/3601510193", 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/3601510193", 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/3601510193", 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "357d2d59-66f0-47b8-87c8-4acdfe62a07d", 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:10:44.534 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 90, 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-15T00:10:44.535 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:10:44.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2024-09-15T00:10:44.537 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2024-09-15T00:10:44.538 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3222759256 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/3222759256", 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/3222759256", 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/3222759256", 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/3222759256", 2024-09-15T00:10:44.539 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "22208f56-d8aa-4a56-a42c-5a990962b860", 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:10:44.540 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 90, 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6809", 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.541 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6811", 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6813", 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6816", 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6822", 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6823", 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6818", 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6821", 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 675428777 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6811/675428777", 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6816/675428777", 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6823/675428777", 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6821/675428777", 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:10:44.545 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "a974629e-9ef0-40a3-94bd-382e0bfa8bb6", 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 0, 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:10:44.546 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 87, 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 90, 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6808", 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6810", 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:10:44.547 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6812", 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6814", 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:10:44.548 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6819", 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6820", 2024-09-15T00:10:44.549 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6815", 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6817", 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3736884138 2024-09-15T00:10:44.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6810/3736884138", 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6814/3736884138", 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6820/3736884138", 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6817/3736884138", 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: "exists" 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.551 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "06b73c2d-466a-4c4e-89fa-31d6eeb1af1d", 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 11, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 90, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T00:10:44.552 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6800", 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6801", 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.553 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6802", 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6803", 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.554 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6806", 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6807", 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6804", 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.79:6805", 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2235671634 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.556 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.79:6801/2235671634", 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.79:6803/2235671634", 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.79:6807/2235671634", 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.79:6805/2235671634", 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2024-09-15T00:10:44.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 65536, 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.730107+0000", 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 56 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:10:44.558 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.365297+0000", 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:10:44.559 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.868500+0000", 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:40.788781+0000", 2024-09-15T00:10:44.560 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.892143+0000", 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 90 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540701547738038271, 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T00:08:39.990037+0000", 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.1d", 2024-09-15T00:10:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 4, 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 2 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.1e", 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 3, 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 4 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2024-09-15T00:10:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [ 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: "snaps": [ 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: "begin": 33, 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: "length": 11 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [ 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: "snaps": [ 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: "begin": 33, 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: "length": 1 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2024-09-15T00:10:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2024-09-15T00:10:44.567 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2024-09-15T00:10:44.567 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2024-09-15T00:10:44.567 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2024-09-15T00:10:44.567 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2024-09-15T00:10:44.567 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2024-09-15T00:10:44.567 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2024-09-15T00:10:44.567 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:44.567 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:44.569 INFO:tasks.thrashosds.thrasher:j is {'epoch': 105, 'fsid': '4f532d31-28c1-4236-a74e-6392960baf2b', 'created': '2024-09-15T00:08:32.007191+0000', 'modified': '2024-09-15T00:10:44.258707+0000', 'last_up_change': '2024-09-15T00:10:28.163783+0000', 'last_in_change': '2024-09-15T00:10:22.135739+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T00:08:42.896076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T00:08:49.173060+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-15T00:08:58.879058+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 43, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8700000047683716, 'score_stable': 1.8700000047683716, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '56600dc2-b95d-47a7-b2b1-5ddc54c08341', 'up': 0, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 56, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 1440638768}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 1440638768}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 1440638768}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 1440638768}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 1440638768}]}, 'public_addr': '172.21.15.27:6817/1440638768', 'cluster_addr': '172.21.15.27:6819/1440638768', 'heartbeat_back_addr': '172.21.15.27:6823/1440638768', 'heartbeat_front_addr': '172.21.15.27:6821/1440638768', 'state': ['exists']}, {'osd': 1, 'uuid': 'ea8f0155-8472-4e87-89fd-7716d05065d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 3601510193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 3601510193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 3601510193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 3601510193}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 3601510193}]}, 'public_addr': '172.21.15.27:6809/3601510193', 'cluster_addr': '172.21.15.27:6811/3601510193', 'heartbeat_back_addr': '172.21.15.27:6815/3601510193', 'heartbeat_front_addr': '172.21.15.27:6813/3601510193', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '357d2d59-66f0-47b8-87c8-4acdfe62a07d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 3222759256}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 3222759256}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 3222759256}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 3222759256}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 3222759256}]}, 'public_addr': '172.21.15.27:6801/3222759256', 'cluster_addr': '172.21.15.27:6803/3222759256', 'heartbeat_back_addr': '172.21.15.27:6807/3222759256', 'heartbeat_front_addr': '172.21.15.27:6805/3222759256', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '22208f56-d8aa-4a56-a42c-5a990962b860', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6809', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 675428777}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6813', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6816', 'nonce': 675428777}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 675428777}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 675428777}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 675428777}]}, 'public_addr': '172.21.15.79:6811/675428777', 'cluster_addr': '172.21.15.79:6816/675428777', 'heartbeat_back_addr': '172.21.15.79:6823/675428777', 'heartbeat_front_addr': '172.21.15.79:6821/675428777', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a974629e-9ef0-40a3-94bd-382e0bfa8bb6', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 87, 'down_at': 90, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6810', 'nonce': 3736884138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6814', 'nonce': 3736884138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6819', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6820', 'nonce': 3736884138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6815', 'nonce': 3736884138}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 3736884138}]}, 'public_addr': '172.21.15.79:6810/3736884138', 'cluster_addr': '172.21.15.79:6814/3736884138', 'heartbeat_back_addr': '172.21.15.79:6820/3736884138', 'heartbeat_front_addr': '172.21.15.79:6817/3736884138', 'state': ['exists']}, {'osd': 5, 'uuid': '06b73c2d-466a-4c4e-89fa-31d6eeb1af1d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 2235671634}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 2235671634}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 2235671634}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 2235671634}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 2235671634}]}, 'public_addr': '172.21.15.79:6801/2235671634', 'cluster_addr': '172.21.15.79:6803/2235671634', 'heartbeat_back_addr': '172.21.15.79:6807/2235671634', 'heartbeat_front_addr': '172.21.15.79:6805/2235671634', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.730107+0000', 'dead_epoch': 56}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.365297+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.868500+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:40.788781+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.892143+0000', 'dead_epoch': 90}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T00:08:39.990037+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1d', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.1e', 'mappings': [{'from': 3, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 11}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T00:10:44.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:10:44.570 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:44.567+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:44.592 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:44.592 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:44.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:44.670 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:44.668+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:44.704 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:44.704 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:44.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:44.773 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:44.772+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:44.821 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:44.821 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:44.830 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:44.899 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:10:44.900 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:10:44.901 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:44.898+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:44.911 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-15T00:10:44.872411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1262009020,"num_objects":1255,"num_object_clones":366,"num_object_copies":2510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":555,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1255,"num_whiteouts":340,"num_read":246808,"num_read_kb":1391228,"num_write":20253,"num_write_kb":430112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1093,"num_bytes_recovered":91226504,"num_keys_recovered":138,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17294,"ondisk_log_size":17294,"up":64,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":304340,"kb_used_data":49776,"kb_used_omap":7,"kb_used_meta":254392,"kb_avail":471554860,"statfs":{"total":483183820800,"available":482872176640,"internally_reserved":0,"allocated":50970624,"data_stored":413416099,"data_compressed":13107116,"data_compressed_allocated":28151808,"data_compressed_original":395499008,"omap_allocated":7973,"internal_metadata":260497627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-79063960,"num_objects":-98,"num_object_clones":-7,"num_object_copies":-196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-98,"num_whiteouts":6,"num_read":1523,"num_read_kb":1299,"num_write":269,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000771"},"pg_stats":[{"pgid":"3.1f","version":"103'452","reported_seq":11098,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.352851+0000","last_change":"2024-09-15T00:10:42.261725+0000","last_active":"2024-09-15T00:10:42.352851+0000","last_peered":"2024-09-15T00:10:42.352851+0000","last_clean":"2024-09-15T00:10:42.352851+0000","last_became_active":"2024-09-15T00:10:26.157785+0000","last_became_peered":"2024-09-15T00:10:26.157785+0000","last_unstale":"2024-09-15T00:10:42.352851+0000","last_undegraded":"2024-09-15T00:10:42.352851+0000","last_fullsized":"2024-09-15T00:10:42.352851+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:59:06.045514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000690856,"stat_sum":{"num_bytes":66940993,"num_objects":41,"num_object_clones":13,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":13,"num_read":11730,"num_read_kb":64289,"num_write":488,"num_write_kb":14114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.1e","version":"103'449","reported_seq":5132,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.259819+0000","last_change":"2024-09-15T00:10:29.185445+0000","last_active":"2024-09-15T00:10:42.259819+0000","last_peered":"2024-09-15T00:10:42.259819+0000","last_clean":"2024-09-15T00:10:27.166534+0000","last_became_active":"2024-09-15T00:10:29.185445+0000","last_became_peered":"2024-09-15T00:10:29.185445+0000","last_unstale":"2024-09-15T00:10:42.259819+0000","last_undegraded":"2024-09-15T00:10:29.182217+0000","last_fullsized":"2024-09-15T00:10:29.182138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:47:00.995148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031880599999999998,"stat_sum":{"num_bytes":36686446,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":6057,"num_read_kb":40329,"num_write":357,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":47}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"103'700","reported_seq":5061,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.266832+0000","last_change":"2024-09-15T00:10:42.266832+0000","last_active":"2024-09-15T00:10:42.266832+0000","last_peered":"2024-09-15T00:10:42.266832+0000","last_clean":"2024-09-15T00:10:42.266832+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:10:42.266832+0000","last_undegraded":"2024-09-15T00:10:42.266832+0000","last_fullsized":"2024-09-15T00:10:42.266832+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:37:57.672578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062882499999999996,"stat_sum":{"num_bytes":24167499,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":9,"num_read":8067,"num_read_kb":36842,"num_write":881,"num_write_kb":5222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.1c","version":"103'440","reported_seq":5596,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.427330+0000","last_change":"2024-09-15T00:10:29.187518+0000","last_active":"2024-09-15T00:10:42.427330+0000","last_peered":"2024-09-15T00:10:42.427330+0000","last_clean":"2024-09-15T00:10:27.916447+0000","last_became_active":"2024-09-15T00:10:29.187518+0000","last_became_peered":"2024-09-15T00:10:29.187518+0000","last_unstale":"2024-09-15T00:10:42.427330+0000","last_undegraded":"2024-09-15T00:10:29.186192+0000","last_fullsized":"2024-09-15T00:10:29.186091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:35:16.721734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090320299999999999,"stat_sum":{"num_bytes":36139061,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":6150,"num_read_kb":39434,"num_write":523,"num_write_kb":7845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":31}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"103'455","reported_seq":10338,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.261313+0000","last_change":"2024-09-15T00:10:29.186025+0000","last_active":"2024-09-15T00:10:42.261313+0000","last_peered":"2024-09-15T00:10:42.261313+0000","last_clean":"2024-09-15T00:10:26.811713+0000","last_became_active":"2024-09-15T00:10:29.186025+0000","last_became_peered":"2024-09-15T00:10:29.186025+0000","last_unstale":"2024-09-15T00:10:42.261313+0000","last_undegraded":"2024-09-15T00:10:29.182935+0000","last_fullsized":"2024-09-15T00:10:29.182814+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:13.784047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048451499999999998,"stat_sum":{"num_bytes":88586330,"num_objects":47,"num_object_clones":16,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":10,"num_read":11051,"num_read_kb":77852,"num_write":541,"num_write_kb":13317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":47}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"102'464","reported_seq":11274,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.444625+0000","last_change":"2024-09-15T00:10:29.193290+0000","last_active":"2024-09-15T00:10:42.444625+0000","last_peered":"2024-09-15T00:10:42.444625+0000","last_clean":"2024-09-15T00:10:27.001856+0000","last_became_active":"2024-09-15T00:10:29.193290+0000","last_became_peered":"2024-09-15T00:10:29.193290+0000","last_unstale":"2024-09-15T00:10:42.444625+0000","last_undegraded":"2024-09-15T00:10:29.188271+0000","last_fullsized":"2024-09-15T00:10:29.188181+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:18:11.129256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085056300000000004,"stat_sum":{"num_bytes":24678545,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":6,"num_read":13416,"num_read_kb":40140,"num_write":397,"num_write_kb":8743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":35}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"103'607","reported_seq":10036,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.439983+0000","last_change":"2024-09-15T00:10:29.190656+0000","last_active":"2024-09-15T00:10:42.439983+0000","last_peered":"2024-09-15T00:10:42.439983+0000","last_clean":"2024-09-15T00:10:27.478500+0000","last_became_active":"2024-09-15T00:10:29.190656+0000","last_became_peered":"2024-09-15T00:10:29.190656+0000","last_unstale":"2024-09-15T00:10:42.439983+0000","last_undegraded":"2024-09-15T00:10:29.188242+0000","last_fullsized":"2024-09-15T00:10:29.188136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:08:10.687207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055705900000000005,"stat_sum":{"num_bytes":33047635,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":8,"num_read":12824,"num_read_kb":54738,"num_write":466,"num_write_kb":22198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"103'558","reported_seq":8136,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.358620+0000","last_change":"2024-09-15T00:10:42.260051+0000","last_active":"2024-09-15T00:10:42.358620+0000","last_peered":"2024-09-15T00:10:42.358620+0000","last_clean":"2024-09-15T00:10:42.358620+0000","last_became_active":"2024-09-15T00:10:26.158999+0000","last_became_peered":"2024-09-15T00:10:26.158999+0000","last_unstale":"2024-09-15T00:10:42.358620+0000","last_undegraded":"2024-09-15T00:10:42.358620+0000","last_fullsized":"2024-09-15T00:10:42.358620+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:26:27.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065909400000000004,"stat_sum":{"num_bytes":77788206,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":10,"num_read":9012,"num_read_kb":64244,"num_write":638,"num_write_kb":14227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.17","version":"103'730","reported_seq":14447,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.457528+0000","last_change":"2024-09-15T00:10:42.267879+0000","last_active":"2024-09-15T00:10:42.457528+0000","last_peered":"2024-09-15T00:10:42.457528+0000","last_clean":"2024-09-15T00:10:42.457528+0000","last_became_active":"2024-09-15T00:10:26.158796+0000","last_became_peered":"2024-09-15T00:10:26.158796+0000","last_unstale":"2024-09-15T00:10:42.457528+0000","last_undegraded":"2024-09-15T00:10:42.457528+0000","last_fullsized":"2024-09-15T00:10:42.457528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:35.682786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014923989999999999,"stat_sum":{"num_bytes":38551114,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":12,"num_read":16339,"num_read_kb":71602,"num_write":806,"num_write_kb":44615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.16","version":"103'743","reported_seq":7638,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.362434+0000","last_change":"2024-09-15T00:10:42.267170+0000","last_active":"2024-09-15T00:10:42.362434+0000","last_peered":"2024-09-15T00:10:42.362434+0000","last_clean":"2024-09-15T00:10:42.362434+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:10:42.362434+0000","last_undegraded":"2024-09-15T00:10:42.362434+0000","last_fullsized":"2024-09-15T00:10:42.362434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:20:57.519439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085203399999999999,"stat_sum":{"num_bytes":41127637,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":9730,"num_read_kb":33542,"num_write":919,"num_write_kb":5131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.15","version":"103'399","reported_seq":6409,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.261222+0000","last_change":"2024-09-15T00:10:29.186127+0000","last_active":"2024-09-15T00:10:42.261222+0000","last_peered":"2024-09-15T00:10:42.261222+0000","last_clean":"2024-09-15T00:10:27.166665+0000","last_became_active":"2024-09-15T00:10:29.186127+0000","last_became_peered":"2024-09-15T00:10:29.186127+0000","last_unstale":"2024-09-15T00:10:42.261222+0000","last_undegraded":"2024-09-15T00:10:29.182954+0000","last_fullsized":"2024-09-15T00:10:29.182765+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:44:52.020431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044338899999999998,"stat_sum":{"num_bytes":42869321,"num_objects":31,"num_object_clones":10,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":6106,"num_read_kb":61941,"num_write":476,"num_write_kb":29389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":31}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"103'539","reported_seq":8079,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.374357+0000","last_change":"2024-09-15T00:10:29.185951+0000","last_active":"2024-09-15T00:10:42.374357+0000","last_peered":"2024-09-15T00:10:42.374357+0000","last_clean":"2024-09-15T00:10:27.166707+0000","last_became_active":"2024-09-15T00:10:29.185951+0000","last_became_peered":"2024-09-15T00:10:29.185951+0000","last_unstale":"2024-09-15T00:10:42.374357+0000","last_undegraded":"2024-09-15T00:10:29.182777+0000","last_fullsized":"2024-09-15T00:10:29.182693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:28:46.136954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079073999999999995,"stat_sum":{"num_bytes":52198416,"num_objects":34,"num_object_clones":13,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":9,"num_read":9432,"num_read_kb":45248,"num_write":385,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":34}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":190,"reported_epoch":103,"state":"active+undersized","last_fresh":"2024-09-15T00:10:42.261182+0000","last_change":"2024-09-15T00:10:29.186634+0000","last_active":"2024-09-15T00:10:42.261182+0000","last_peered":"2024-09-15T00:10:42.261182+0000","last_clean":"2024-09-15T00:10:27.166812+0000","last_became_active":"2024-09-15T00:10:29.186634+0000","last_became_peered":"2024-09-15T00:10:29.186634+0000","last_unstale":"2024-09-15T00:10:42.261182+0000","last_undegraded":"2024-09-15T00:10:42.261182+0000","last_fullsized":"2024-09-15T00:10:29.183024+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:09:52.645547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"103'379","reported_seq":3153,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.355625+0000","last_change":"2024-09-15T00:10:29.193339+0000","last_active":"2024-09-15T00:10:42.355625+0000","last_peered":"2024-09-15T00:10:42.355625+0000","last_clean":"2024-09-15T00:10:27.478666+0000","last_became_active":"2024-09-15T00:10:29.193339+0000","last_became_peered":"2024-09-15T00:10:29.193339+0000","last_unstale":"2024-09-15T00:10:42.355625+0000","last_undegraded":"2024-09-15T00:10:29.188409+0000","last_fullsized":"2024-09-15T00:10:29.188325+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:20:32.798152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083591999999999998,"stat_sum":{"num_bytes":41024512,"num_objects":45,"num_object_clones":15,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":16,"num_read":2834,"num_read_kb":21154,"num_write":390,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":45}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":188,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.257786+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:10:42.257786+0000","last_peered":"2024-09-15T00:10:42.257786+0000","last_clean":"2024-09-15T00:10:42.257786+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:10:42.257786+0000","last_undegraded":"2024-09-15T00:10:42.257786+0000","last_fullsized":"2024-09-15T00:10:42.257786+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"103'278","reported_seq":3304,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.382608+0000","last_change":"2024-09-15T00:10:42.260628+0000","last_active":"2024-09-15T00:10:42.382608+0000","last_peered":"2024-09-15T00:10:42.382608+0000","last_clean":"2024-09-15T00:10:42.382608+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:10:42.382608+0000","last_undegraded":"2024-09-15T00:10:42.382608+0000","last_fullsized":"2024-09-15T00:10:42.382608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:13:06.169249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082340199999999999,"stat_sum":{"num_bytes":14476862,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":3135,"num_read_kb":36812,"num_write":309,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":197,"reported_epoch":103,"state":"active+undersized","last_fresh":"2024-09-15T00:10:42.261310+0000","last_change":"2024-09-15T00:10:29.186795+0000","last_active":"2024-09-15T00:10:42.261310+0000","last_peered":"2024-09-15T00:10:42.261310+0000","last_clean":"2024-09-15T00:10:27.166762+0000","last_became_active":"2024-09-15T00:10:29.186795+0000","last_became_peered":"2024-09-15T00:10:29.186795+0000","last_unstale":"2024-09-15T00:10:42.261310+0000","last_undegraded":"2024-09-15T00:10:42.261310+0000","last_fullsized":"2024-09-15T00:10:29.183158+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:35.279994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"103'685","reported_seq":6688,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.258340+0000","last_change":"2024-09-15T00:10:42.258340+0000","last_active":"2024-09-15T00:10:42.258340+0000","last_peered":"2024-09-15T00:10:42.258340+0000","last_clean":"2024-09-15T00:10:42.258340+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:10:42.258340+0000","last_undegraded":"2024-09-15T00:10:42.258340+0000","last_fullsized":"2024-09-15T00:10:42.258340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:47:18.392711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072767100000000005,"stat_sum":{"num_bytes":41891011,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":9396,"num_read_kb":67289,"num_write":754,"num_write_kb":50193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":197,"reported_epoch":103,"state":"active+undersized","last_fresh":"2024-09-15T00:10:42.257634+0000","last_change":"2024-09-15T00:10:29.193271+0000","last_active":"2024-09-15T00:10:42.257634+0000","last_peered":"2024-09-15T00:10:42.257634+0000","last_clean":"2024-09-15T00:10:27.478681+0000","last_became_active":"2024-09-15T00:10:29.193271+0000","last_became_peered":"2024-09-15T00:10:29.193271+0000","last_unstale":"2024-09-15T00:10:42.257634+0000","last_undegraded":"2024-09-15T00:10:42.257634+0000","last_fullsized":"2024-09-15T00:10:29.188315+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:03:45.984799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"103'524","reported_seq":5547,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.459650+0000","last_change":"2024-09-15T00:10:42.266937+0000","last_active":"2024-09-15T00:10:42.459650+0000","last_peered":"2024-09-15T00:10:42.459650+0000","last_clean":"2024-09-15T00:10:42.459650+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:10:42.459650+0000","last_undegraded":"2024-09-15T00:10:42.459650+0000","last_fullsized":"2024-09-15T00:10:42.459650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:11:15.349601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061279599999999998,"stat_sum":{"num_bytes":31078030,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":12,"num_read":6319,"num_read_kb":34625,"num_write":611,"num_write_kb":6840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.3","version":"103'484","reported_seq":5183,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.398140+0000","last_change":"2024-09-15T00:10:29.186299+0000","last_active":"2024-09-15T00:10:42.398140+0000","last_peered":"2024-09-15T00:10:42.398140+0000","last_clean":"2024-09-15T00:09:42.597919+0000","last_became_active":"2024-09-15T00:10:29.186299+0000","last_became_peered":"2024-09-15T00:10:29.186299+0000","last_unstale":"2024-09-15T00:10:42.398140+0000","last_undegraded":"2024-09-15T00:10:29.182626+0000","last_fullsized":"2024-09-15T00:10:29.182540+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:01.337063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037381900000000001,"stat_sum":{"num_bytes":57959463,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":6868,"num_read_kb":26224,"num_write":407,"num_write_kb":3319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":38}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":191,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.266158+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:10:42.266158+0000","last_peered":"2024-09-15T00:10:42.266158+0000","last_clean":"2024-09-15T00:10:42.266158+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:10:42.266158+0000","last_undegraded":"2024-09-15T00:10:42.266158+0000","last_fullsized":"2024-09-15T00:10:42.266158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"103'634","reported_seq":6786,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.258337+0000","last_change":"2024-09-15T00:10:42.258337+0000","last_active":"2024-09-15T00:10:42.258337+0000","last_peered":"2024-09-15T00:10:42.258337+0000","last_clean":"2024-09-15T00:10:42.258337+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:10:42.258337+0000","last_undegraded":"2024-09-15T00:10:42.258337+0000","last_fullsized":"2024-09-15T00:10:42.258337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:41:52.929104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001052409,"stat_sum":{"num_bytes":16828254,"num_objects":45,"num_object_clones":15,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":13,"num_read":7688,"num_read_kb":54027,"num_write":870,"num_write_kb":19879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":189,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.259585+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:10:42.259585+0000","last_peered":"2024-09-15T00:10:42.259585+0000","last_clean":"2024-09-15T00:10:42.259585+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:10:42.259585+0000","last_undegraded":"2024-09-15T00:10:42.259585+0000","last_fullsized":"2024-09-15T00:10:42.259585+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"101'588","reported_seq":4199,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.259373+0000","last_change":"2024-09-15T00:10:29.182787+0000","last_active":"2024-09-15T00:10:42.259373+0000","last_peered":"2024-09-15T00:10:42.259373+0000","last_clean":"2024-09-15T00:10:26.913952+0000","last_became_active":"2024-09-15T00:10:29.182787+0000","last_became_peered":"2024-09-15T00:10:29.182787+0000","last_unstale":"2024-09-15T00:10:42.259373+0000","last_undegraded":"2024-09-15T00:10:29.182013+0000","last_fullsized":"2024-09-15T00:10:29.181923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:04:38.146379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000355144,"stat_sum":{"num_bytes":16774778,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":5253,"num_read_kb":27105,"num_write":851,"num_write_kb":8032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":1775375,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":32}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":188,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.260696+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:10:42.260696+0000","last_peered":"2024-09-15T00:10:42.260696+0000","last_clean":"2024-09-15T00:10:42.260696+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:10:42.260696+0000","last_undegraded":"2024-09-15T00:10:42.260696+0000","last_fullsized":"2024-09-15T00:10:42.260696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"103'416","reported_seq":6444,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.365903+0000","last_change":"2024-09-15T00:10:42.258940+0000","last_active":"2024-09-15T00:10:42.365903+0000","last_peered":"2024-09-15T00:10:42.365903+0000","last_clean":"2024-09-15T00:10:42.365903+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:10:42.365903+0000","last_undegraded":"2024-09-15T00:10:42.365903+0000","last_fullsized":"2024-09-15T00:10:42.365903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:17.749881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094728599999999998,"stat_sum":{"num_bytes":53609027,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":5987,"num_read_kb":25269,"num_write":362,"num_write_kb":5274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"2.3","version":"16'1","reported_seq":190,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.266156+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:10:42.266156+0000","last_peered":"2024-09-15T00:10:42.266156+0000","last_clean":"2024-09-15T00:10:42.266156+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:10:42.266156+0000","last_undegraded":"2024-09-15T00:10:42.266156+0000","last_fullsized":"2024-09-15T00:10:42.266156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":354,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.260729+0000","last_change":"2024-09-15T00:10:26.160336+0000","last_active":"2024-09-15T00:10:42.260729+0000","last_peered":"2024-09-15T00:10:42.260729+0000","last_clean":"2024-09-15T00:10:42.260729+0000","last_became_active":"2024-09-15T00:10:26.159969+0000","last_became_peered":"2024-09-15T00:10:26.159969+0000","last_unstale":"2024-09-15T00:10:42.260729+0000","last_undegraded":"2024-09-15T00:10:42.260729+0000","last_fullsized":"2024-09-15T00:10:42.260729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:40:41.381426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"103'460","reported_seq":10461,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.431506+0000","last_change":"2024-09-15T00:10:42.260167+0000","last_active":"2024-09-15T00:10:42.431506+0000","last_peered":"2024-09-15T00:10:42.431506+0000","last_clean":"2024-09-15T00:10:42.431506+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:10:42.431506+0000","last_undegraded":"2024-09-15T00:10:42.431506+0000","last_fullsized":"2024-09-15T00:10:42.431506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:09:35.539109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064170999999999998,"stat_sum":{"num_bytes":63458352,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":9,"num_read":12057,"num_read_kb":87531,"num_write":536,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.9","version":"103'429","reported_seq":8779,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.355448+0000","last_change":"2024-09-15T00:10:29.185395+0000","last_active":"2024-09-15T00:10:42.355448+0000","last_peered":"2024-09-15T00:10:42.355448+0000","last_clean":"2024-09-15T00:10:27.025173+0000","last_became_active":"2024-09-15T00:10:29.185395+0000","last_became_peered":"2024-09-15T00:10:29.185395+0000","last_unstale":"2024-09-15T00:10:42.355448+0000","last_undegraded":"2024-09-15T00:10:29.182147+0000","last_fullsized":"2024-09-15T00:10:29.182067+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:12.704204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031568600000000003,"stat_sum":{"num_bytes":46475353,"num_objects":54,"num_object_clones":19,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":12,"num_read":9105,"num_read_kb":49528,"num_write":380,"num_write_kb":13137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":54}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"103'308","reported_seq":2160,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.261229+0000","last_change":"2024-09-15T00:10:29.186259+0000","last_active":"2024-09-15T00:10:42.261229+0000","last_peered":"2024-09-15T00:10:42.261229+0000","last_clean":"2024-09-15T00:10:27.166504+0000","last_became_active":"2024-09-15T00:10:29.186259+0000","last_became_peered":"2024-09-15T00:10:29.186259+0000","last_unstale":"2024-09-15T00:10:42.261229+0000","last_undegraded":"2024-09-15T00:10:29.182670+0000","last_fullsized":"2024-09-15T00:10:29.182562+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:50.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035268300000000001,"stat_sum":{"num_bytes":3242659,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":2471,"num_read_kb":16261,"num_write":290,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"101'283","reported_seq":3403,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.257573+0000","last_change":"2024-09-15T00:10:29.193323+0000","last_active":"2024-09-15T00:10:42.257573+0000","last_peered":"2024-09-15T00:10:42.257573+0000","last_clean":"2024-09-15T00:10:27.478584+0000","last_became_active":"2024-09-15T00:10:29.193323+0000","last_became_peered":"2024-09-15T00:10:29.193323+0000","last_unstale":"2024-09-15T00:10:42.257573+0000","last_undegraded":"2024-09-15T00:10:29.188358+0000","last_fullsized":"2024-09-15T00:10:29.188279+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:11:25.898421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071565300000000001,"stat_sum":{"num_bytes":40249464,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":3241,"num_read_kb":33162,"num_write":264,"num_write_kb":6935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":31}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"103'1011","reported_seq":5514,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.406304+0000","last_change":"2024-09-15T00:10:42.267714+0000","last_active":"2024-09-15T00:10:42.406304+0000","last_peered":"2024-09-15T00:10:42.406304+0000","last_clean":"2024-09-15T00:10:42.406304+0000","last_became_active":"2024-09-15T00:10:26.159829+0000","last_became_peered":"2024-09-15T00:10:26.159829+0000","last_unstale":"2024-09-15T00:10:42.406304+0000","last_undegraded":"2024-09-15T00:10:42.406304+0000","last_fullsized":"2024-09-15T00:10:42.406304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1011,"log_dups_size":0,"ondisk_log_size":1011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:07.040838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013936039999999999,"stat_sum":{"num_bytes":31333960,"num_objects":36,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":13,"num_read":8487,"num_read_kb":30843,"num_write":1597,"num_write_kb":11575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.d","version":"103'405","reported_seq":4757,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.262100+0000","last_change":"2024-09-15T00:10:42.262100+0000","last_active":"2024-09-15T00:10:42.262100+0000","last_peered":"2024-09-15T00:10:42.262100+0000","last_clean":"2024-09-15T00:10:42.262100+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:10:42.262100+0000","last_undegraded":"2024-09-15T00:10:42.262100+0000","last_fullsized":"2024-09-15T00:10:42.262100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:50:55.369612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081297300000000004,"stat_sum":{"num_bytes":22487649,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":6044,"num_read_kb":25220,"num_write":394,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.e","version":"103'277","reported_seq":4110,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.358615+0000","last_change":"2024-09-15T00:10:29.188137+0000","last_active":"2024-09-15T00:10:42.358615+0000","last_peered":"2024-09-15T00:10:42.358615+0000","last_clean":"2024-09-15T00:10:27.916508+0000","last_became_active":"2024-09-15T00:10:29.188137+0000","last_became_peered":"2024-09-15T00:10:29.188137+0000","last_unstale":"2024-09-15T00:10:42.358615+0000","last_undegraded":"2024-09-15T00:10:29.186387+0000","last_fullsized":"2024-09-15T00:10:29.186304+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:14:50.933356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010923459999999999,"stat_sum":{"num_bytes":39438037,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":11,"num_read":3787,"num_read_kb":22321,"num_write":275,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":42}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"101'659","reported_seq":4660,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.267968+0000","last_change":"2024-09-15T00:10:42.267968+0000","last_active":"2024-09-15T00:10:42.267968+0000","last_peered":"2024-09-15T00:10:42.267968+0000","last_clean":"2024-09-15T00:10:42.267968+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:10:42.267968+0000","last_undegraded":"2024-09-15T00:10:42.267968+0000","last_fullsized":"2024-09-15T00:10:42.267968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:42.495376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014702859999999999,"stat_sum":{"num_bytes":8864647,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":8,"num_read":6594,"num_read_kb":21889,"num_write":734,"num_write_kb":6478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.10","version":"103'299","reported_seq":5716,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.360893+0000","last_change":"2024-09-15T00:10:42.266633+0000","last_active":"2024-09-15T00:10:42.360893+0000","last_peered":"2024-09-15T00:10:42.360893+0000","last_clean":"2024-09-15T00:10:42.360893+0000","last_became_active":"2024-09-15T00:10:26.161430+0000","last_became_peered":"2024-09-15T00:10:26.161430+0000","last_unstale":"2024-09-15T00:10:42.360893+0000","last_undegraded":"2024-09-15T00:10:42.360893+0000","last_fullsized":"2024-09-15T00:10:42.360893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:29:19.659529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044715199999999998,"stat_sum":{"num_bytes":32620064,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":10,"num_read":5076,"num_read_kb":32661,"num_write":363,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.11","version":"103'1137","reported_seq":4368,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-15T00:10:42.459114+0000","last_change":"2024-09-15T00:10:29.183668+0000","last_active":"2024-09-15T00:10:42.459114+0000","last_peered":"2024-09-15T00:10:42.459114+0000","last_clean":"2024-09-15T00:10:27.166497+0000","last_became_active":"2024-09-15T00:10:29.183668+0000","last_became_peered":"2024-09-15T00:10:29.183668+0000","last_unstale":"2024-09-15T00:10:42.459114+0000","last_undegraded":"2024-09-15T00:10:29.182595+0000","last_fullsized":"2024-09-15T00:10:29.182490+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:07:20.000975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067808099999999997,"stat_sum":{"num_bytes":39700556,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":7812,"num_read_kb":21140,"num_write":1765,"num_write_kb":5416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":28}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"103'1175","reported_seq":9952,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.360905+0000","last_change":"2024-09-15T00:10:42.261692+0000","last_active":"2024-09-15T00:10:42.360905+0000","last_peered":"2024-09-15T00:10:42.360905+0000","last_clean":"2024-09-15T00:10:42.360905+0000","last_became_active":"2024-09-15T00:10:26.160011+0000","last_became_peered":"2024-09-15T00:10:26.160011+0000","last_unstale":"2024-09-15T00:10:42.360905+0000","last_undegraded":"2024-09-15T00:10:42.360905+0000","last_fullsized":"2024-09-15T00:10:42.360905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1175,"log_dups_size":0,"ondisk_log_size":1175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:25:36.063164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065642900000000004,"stat_sum":{"num_bytes":86600824,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":12,"num_read":12770,"num_read_kb":112368,"num_write":1848,"num_write_kb":43527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.13","version":"94'207","reported_seq":2190,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-15T00:10:42.261830+0000","last_change":"2024-09-15T00:10:42.262123+0000","last_active":"2024-09-15T00:10:42.261830+0000","last_peered":"2024-09-15T00:10:42.261830+0000","last_clean":"2024-09-15T00:10:42.261830+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:10:42.261830+0000","last_undegraded":"2024-09-15T00:10:42.261830+0000","last_fullsized":"2024-09-15T00:10:42.261830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:50:51.375627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083307499999999996,"stat_sum":{"num_bytes":10523928,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":8,"num_read":1854,"num_read_kb":15498,"num_write":215,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1261418633,"num_objects":1251,"num_object_clones":366,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":555,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1251,"num_whiteouts":340,"num_read":246692,"num_read_kb":1391128,"num_write":20092,"num_write_kb":427694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1091,"num_bytes_recovered":90636136,"num_keys_recovered":138,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53309440,"data_stored":494727979,"data_compressed":15168015,"data_compressed_allocated":33755136,"data_compressed_original":476391424,"omap_allocated":0,"internal_metadata":0},"log_size":17174,"ondisk_log_size":17174,"up":49,"acting":49,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":13,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":26618,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640283,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60928,"kb_used_data":7524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310912,"statfs":{"total":96636764160,"available":96574373888,"internally_reserved":0,"allocated":7704576,"data_stored":54294069,"data_compressed":1856648,"data_compressed_allocated":3919872,"data_compressed_original":51640832,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640279,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58464,"kb_used_data":9228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94313376,"statfs":{"total":96636764160,"available":96576897024,"internally_reserved":0,"allocated":9449472,"data_stored":72878443,"data_compressed":2296039,"data_compressed_allocated":4767744,"data_compressed_original":69066240,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82536,"kb_used_data":11056,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94289304,"statfs":{"total":96636764160,"available":96552247296,"internally_reserved":0,"allocated":11321344,"data_stored":91366575,"data_compressed":3021716,"data_compressed_allocated":6422528,"data_compressed_original":87318016,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[1,2,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640283,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60756,"kb_used_data":11508,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94311084,"statfs":{"total":96636764160,"available":96574550016,"internally_reserved":0,"allocated":11784192,"data_stored":98289981,"data_compressed":3162604,"data_compressed_allocated":6733824,"data_compressed_original":94215168,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41656,"kb_used_data":10460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330184,"statfs":{"total":96636764160,"available":96594108416,"internally_reserved":0,"allocated":10711040,"data_stored":96587031,"data_compressed":2770109,"data_compressed_allocated":6307840,"data_compressed_original":93258752,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9273344,"data_stored":90570113,"data_compressed":2315776,"data_compressed_allocated":6262784,"data_compressed_original":87402496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9449472,"data_stored":95495924,"data_compressed":2760547,"data_compressed_allocated":6270976,"data_compressed_original":92668928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9945088,"data_stored":93420995,"data_compressed":2972699,"data_compressed_allocated":6287360,"data_compressed_original":89996800,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10047488,"data_stored":90275487,"data_compressed":3013188,"data_compressed_allocated":6385664,"data_compressed_original":86744576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8466432,"data_stored":72479240,"data_compressed":2296039,"data_compressed_allocated":4767744,"data_compressed_original":69066240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6127616,"data_stored":52486220,"data_compressed":1809766,"data_compressed_allocated":3780608,"data_compressed_original":50512384,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:10:44.914 INFO:tasks.thrashosds.thrasher:Setting 3.12 to [0, 5] 2024-09-15T00:10:44.914 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.12', '0', '5'] 2024-09-15T00:10:44.914 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.12 0 5 2024-09-15T00:10:44.974 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:44.973+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:44.978 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:44.978 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:44.988 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:45.076 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:45.075+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:45.137 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:45.137 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:45.146 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:45.177 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:45.175+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:45.267 INFO:teuthology.orchestra.run.smithi027.stderr:set 3.12 pg_upmap mapping to [0,5] 2024-09-15T00:10:45.270 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:10:45.282 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:45.281+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:45.304 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:45.305 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:45.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:45.398 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:45.397+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:45.464 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:45.518 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:45.521 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:45.521 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:45.519+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:45.621 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:45.619+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:45.667 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:45.667 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:45.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:45.723 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:45.720+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:45.781 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:45.781 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:45.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:45.833 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:45.832+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:45.892 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:45.893 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:45.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:45.942 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:45.940+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:46.025 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:46.024+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:46.043 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:46.044 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:46.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:46.103 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:46.137 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:46.135+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:46.196 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:46.197 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:46.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:46.214 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:46.226 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:46.224+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:46.231 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:46.248 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:46.248 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:46.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:46.265 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:46.265 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:46.286 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:46.286 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:46.289 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:46.289 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:46.331 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:46.329+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:46.351 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:46.351 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:46.351 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:46.360 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:46.428 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:46.426+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:46.466 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.580816188, 2024-09-15T00:10:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.580777557999994, 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.580751621000005, 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.481 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.580735552999997, 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:46.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.580594457999993, 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.490 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.579737671999993, 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:46.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.579702655999995, 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.579677052999997, 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.579502786999996, 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.579271648000002, 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.578459703999997, 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.578379964000007, 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.578174938999993, 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.577963405999995, 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.575407398999999, 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.573424293000002, 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.573404722000006, 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 44.302661872000002, 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.396910377000001, 2024-09-15T00:10:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.779822936999999, 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:46.559 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:46.560 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:46.559+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:46.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:46.661 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:46.659+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:46.669 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:46.669 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:46.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:46.762 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:46.760+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:46.825 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:46.826 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:46.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:46.863 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:46.862+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:46.943 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:46.943 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913606256999998, 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.944 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:46.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:46.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913586511000005, 2024-09-15T00:10:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913566560999996, 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:46.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:46.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913539060999994, 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.959 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:46.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:46.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:46.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:46.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:46.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:46.964 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913522220000004, 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:46.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:46.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:46.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:46.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:46.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:46.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913506221999995, 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.970 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:46.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:46.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913483233999997, 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.975 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.979 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:46.980 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913457350000002, 2024-09-15T00:10:46.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:46.980 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.980 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.980 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.980 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:46.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:46.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:46.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:46.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:46.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:46.984 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913440922999996, 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.985 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:46.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.989 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913426651999998, 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:46.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.994 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.913417335000005, 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:46.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:46.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.911748322999998, 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:47.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.911713259999999, 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:47.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.911688233000007, 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:47.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:47.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.911663924999999, 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:47.013 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:47.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:47.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.909447541999995, 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.018 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.022 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 76.909418169999995, 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:47.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:47.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:47.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.027 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 28.244914742999999, 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:47.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:47.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:10:47.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:10:47.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:10:47.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.035 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5644.0:745 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.769320868000001, 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227015319999999, 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5644", 2024-09-15T00:10:47.036 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1022470035", 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 745 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171868+0000", 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-06 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171876+0000", 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.593e-06 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171890+0000", 2024-09-15T00:10:47.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4347e-05 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171897+0000", 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8449999999999997e-06 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171949+0000", 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2206000000000001e-05 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:47.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171959+0000", 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7829999999999993e-06 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194277+0000", 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0223181750000001 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194432+0000", 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155329 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.040 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194568+0000", 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013594599999999999 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20982 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.769253178, 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0231241369999999, 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20982 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171935+0000", 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1200000000000003e-07 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171938+0000", 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9550000000000001e-06 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171944+0000", 2024-09-15T00:10:47.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:10:47.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.044 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171948+0000", 2024-09-15T00:10:47.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8519999999999997e-06 2024-09-15T00:10:47.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171977+0000", 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9539999999999998e-05 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171981+0000", 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.478e-06 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194852+0000", 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0228714160000001 2024-09-15T00:10:47.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194908+0000", 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5837000000000001e-05 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.195059+0000", 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015047699999999999 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.046 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.047 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:47.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:47.049 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:47.046+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:47.149 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:47.147+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:47.156 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:47.156 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:47.156 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:47.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:47.249 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:47.247+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:47.272 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:47.272 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:47.272 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:47.272 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:47.272 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:47.281 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:47.292 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:10:47.365 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:47.363+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.414335402000006, 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.414313484000004, 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.414292212000007, 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.412550952000004, 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.412529046000003, 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.412506953999994, 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.468 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:47.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.412475215000001, 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.473 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:47.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.477 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.412177012000001, 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:47.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:47.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:47.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.482 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.482 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.412143306999994, 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:47.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:47.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:47.487 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.407841234000003, 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:47.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.407516364000003, 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.407105959000006, 2024-09-15T00:10:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:47.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:47.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.407051056, 2024-09-15T00:10:47.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:47.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:47.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.407019020999996, 2024-09-15T00:10:47.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:47.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.405541190999998, 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:47.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.517 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.405519475999995, 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:47.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:47.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:47.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:47.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:47.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:47.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:47.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:47.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.524 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.524 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.524 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.405497996999998, 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:47.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:47.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:47.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:47.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.529 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.529 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.403484892999998, 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:47.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.403465252000004, 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:47.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:47.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.403418704000003, 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:47.539 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:47.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:47.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.544 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.544 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.544 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:47.544 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:47.544 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:47.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:47.546 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:47.543+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:47.647 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:47.644+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:47.651 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:47.651 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:47.651 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:47.651 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:47.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:47.747 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:47.745+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:47.762 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:47.762 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:47.762 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:47.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:47.848 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:47.846+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:47.875 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:47.875 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:47.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:47.875 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:47.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.875 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.875 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.960559007000001, 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:47.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:47.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.960538700999997, 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:47.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:47.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:47.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:47.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:47.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:47.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:47.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:47.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.960516761000001, 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:47.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:47.892 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.958658990000004, 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:47.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:47.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.958629311000003, 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.898 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.958606218, 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:47.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:47.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:47.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.908 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.958581045000003, 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:47.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:47.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.913 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.958560403999996, 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:47.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:47.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.958539862999999, 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:47.919 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.958516381000003, 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:47.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:47.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:47.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:47.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:47.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.956652724999998, 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:47.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:47.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.956622955999997, 2024-09-15T00:10:47.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:47.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.941 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.956600776999998, 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.942 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:47.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.946 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.956573527000003, 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.947 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:47.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.956546932999998, 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:47.952 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:47.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:47.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:47.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:47.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:47.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.957 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.956527600000001, 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:47.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:47.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:47.963 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.954479190999997, 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:47.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.954463957999998, 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:47.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:47.974 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.189187439000001, 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:10:47.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.980 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.143096769, 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:47.981 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:47.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:10:47.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:10:47.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:47.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:10:47.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:10:47.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:47.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:10:47.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:10:47.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:47.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:47.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:10:47.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:10:47.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.988 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:47.988 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:47.988 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:47.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:47.990 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:47.988+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:48.090 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:48.089+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:48.133 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:48.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:48.133 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:48.133 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:48.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:48.225 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:48.224+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:48.284 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:48.284 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:48.285 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:48.285 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:48.285 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:48.288 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:10:48.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:48.291 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:48.289+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:48.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:48.392 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:48.390+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:48.409 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:48.429 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:48.430 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:48.447 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:48.447 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:48.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:48.447 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.561884059999997, 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.561845430000005, 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:48.452 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:48.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:48.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.460 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.460 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:48.460 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.561819493000002, 2024-09-15T00:10:48.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:48.460 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:48.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.464 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.561803424999994, 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:48.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.469 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.561662330000004, 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:48.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:48.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.560805544000004, 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:48.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.478 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.560770528000006, 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:48.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.560744924999995, 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:48.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:48.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.560570659000007, 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.488 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.560339519999999, 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.493 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:48.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.559527575999994, 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:48.497 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:48.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:48.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.559447836000004, 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.506 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.559242811000004, 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:48.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:48.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:48.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.511 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.559031278000006, 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:48.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.556475270999996, 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.516 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:48.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:48.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.554492164999999, 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:48.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.554472594000003, 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:48.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.283729743999999, 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:48.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.377978249000002, 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:48.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.760890808999999, 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:48.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:48.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:48.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:48.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:48.546 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:48.547 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:48.547 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:48.548 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:48.548 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:48.548 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:48.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:48.549 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:48.550 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:48.550 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:48.550 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:48.550 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:48.545+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:48.648 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:48.647+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:48.696 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:48.696 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:48.705 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:48.750 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:48.748+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:48.812 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:48.812 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:48.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:48.851 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:48.849+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:48.927 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:48.928 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:48.937 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:48.952 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:48.950+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:49.044 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:49.044 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:49.052 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:49.053 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:49.051+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:49.153 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:49.151+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:49.159 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:49.159 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:49.168 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:49.253 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:49.251+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:49.277 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:49.277 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:49.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:49.354 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:49.352+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:49.406 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:49.406 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:49.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:49.455 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:49.453+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:49.520 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:49.520 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:49.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:49.571 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:49.570+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:49.630 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:49.631 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:49.639 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:49.657 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:49.655+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:49.758 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:49.756+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:49.782 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:49.783 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:49.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:49.875 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:49.874+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:49.935 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:49.935 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:49.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:49.985 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:49.983+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:50.060 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:50.058+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:50.086 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:50.086 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:50.094 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:50.161 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:50.159+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:50.215 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:50.216 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:50.225 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:50.270 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:50.268+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:50.284 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [0, 4] live_osds: [1, 2, 3, 5] 2024-09-15T00:10:50.284 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:10:50.284 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-15T00:10:50.285 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T00:10:50.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T00:10:50.287 INFO:tasks.ceph.osd.4:Started 2024-09-15T00:10:50.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:50.307 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:10:50.341 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:50.341 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:50.351 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:50.362 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:50.361+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:50.399 INFO:teuthology.orchestra.run.smithi079.stderr:no valid command found; 10 closest matches: 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:0 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:1 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:2 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:abort 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:assert 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:config diff 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:config diff get 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:config get 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:config help [] 2024-09-15T00:10:50.400 INFO:teuthology.orchestra.run.smithi079.stderr:config set ... 2024-09-15T00:10:50.401 INFO:teuthology.orchestra.run.smithi079.stderr:admin_socket: invalid command 2024-09-15T00:10:50.402 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:10:50.402 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-15T00:10:50.463 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:50.461+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:50.502 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:50.503 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:50.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:50.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:50.564 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:50.562+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:50.660 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:50.663 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:50.663 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:50.665 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:50.663+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:50.672 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:50.677 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:50.677 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:50.696 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:50.696 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:50.714 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:50.731 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:50.731 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:50.734 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:50.734 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:50.778 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:50.775+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:50.786 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:50.786 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:50.786 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:50.786 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:50.787 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:50.795 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:50.881 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:50.879+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.918777399999996, 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.951 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.918755481999995, 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:50.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:50.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:50.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:50.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:50.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.956 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.918734209999997, 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.916992949999994, 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:50.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:50.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.916971043999993, 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.966 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.970 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.916948951999998, 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:50.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:50.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.916917213000005, 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.975 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:50.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.979 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.916619010000005, 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.916585304999998, 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.912283231999993, 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:50.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:50.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.911958362000007, 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:50.997 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.911547956999996, 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:50.998 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:50.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:51.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:51.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.911493054000005, 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.003 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:51.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.911461019000001, 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:51.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:51.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.011 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.909983189000002, 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:51.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:51.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.909961473999999, 2024-09-15T00:10:51.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.020 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.909939995000002, 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:51.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:51.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:51.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:51.025 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.907926891000002, 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:51.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:51.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.907907249999994, 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:51.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:51.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.034 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 80.907860701999994, 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.035 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:51.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:51.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:51.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:51.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:51.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.039 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:51.039 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:51.039 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:51.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:51.041 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:51.039+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:51.141 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:51.140+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:51.146 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:51.146 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:51.146 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:51.146 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:51.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:51.242 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:51.241+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:51.298 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:51.299 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:51.304 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:10:51.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:51.343 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:51.341+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:51.444 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:51.442+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:51.459 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.545266142000003, 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.545245835999999, 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.545223896000003, 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.543366124999999, 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.543336445999998, 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.543313353000002, 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.543288179999998, 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.543267538999999, 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.543246998000001, 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.543223515999998, 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.54135986, 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.541330090999999, 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.541307912000001, 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.541280661999998, 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.541254068000001, 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.541234735000003, 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.539186325999999, 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.539171093, 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.773894574000003, 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:10:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:10:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.727803903999998, 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:10:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:10:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:10:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:10:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:10:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:10:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:10:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.562 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:51.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:51.564 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:51.562+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:51.664 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:51.663+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:51.708 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:51.709 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:51.710 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:51.708+0000 7ff91860e640 -1 Falling back to public interface 2024-09-15T00:10:51.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:51.766 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:51.764+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:51.820 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:51.821 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:51.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:51.867 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:51.865+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:51.933 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:51.933 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:51.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.048157672000002, 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.934 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:51.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:51.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:51.938 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.048119041999996, 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:51.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.048093105000007, 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:51.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.048077036999999, 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:51.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.047935941999995, 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:51.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.047079155999995, 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.957 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.961 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.047044139999997, 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:51.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:51.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:51.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:51.966 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.047018537, 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:51.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:51.970 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.046844270999998, 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.971 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:51.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:51.975 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.046613132000004, 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:51.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.045801187999999, 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:51.980 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.984 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.045721447999995, 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:51.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.989 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.045516422999995, 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:51.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.994 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.045304889999997, 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:51.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.042748883000002, 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:51.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.040765777000004, 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:52.004 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:52.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:52.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:52.008 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.040746205999994, 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:52.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:52.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:52.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.770003355999997, 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:52.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:52.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:52.017 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.864251861, 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:52.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:52.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:52.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.247164421000001, 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:52.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:52.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:52.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:52.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:52.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:52.027 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:52.029 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:52.029 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:52.026+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:52.133 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:52.128+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:52.144 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:52.145 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:52.145 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:52.145 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:52.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:52.231 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:52.229+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:52.260 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:52.260 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:52.261 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:52.261 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:52.261 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:52.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:52.332 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:52.330+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391977088000004, 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:52.422 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391957341999998, 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:52.427 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:52.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391937392000003, 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:52.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391909892000001, 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:52.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:52.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391893050999997, 2024-09-15T00:10:52.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.445 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391877053000002, 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.446 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:52.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:52.450 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391854065000004, 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:52.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:52.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391828180999994, 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:52.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:52.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391811754000003, 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:52.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:52.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:52.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:52.464 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391797483000005, 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:52.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:52.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391788165999998, 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.469 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:52.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:52.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.390119154000004, 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:52.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.390084091000006, 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.478 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:52.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.390059063999999, 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:52.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.486 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.390034756000006, 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.487 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:52.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.491 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.387818373000002, 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:52.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.387789001000002, 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.496 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:52.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.500 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 33.723285574000002, 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:10:52.501 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:52.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5644.0:745 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.247691699000001, 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227015319999999, 2024-09-15T00:10:52.507 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5644", 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1022470035", 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 745 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:52.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171868+0000", 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-06 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171876+0000", 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.593e-06 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171890+0000", 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4347e-05 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171897+0000", 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8449999999999997e-06 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171949+0000", 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2206000000000001e-05 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171959+0000", 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7829999999999993e-06 2024-09-15T00:10:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194277+0000", 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0223181750000001 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194432+0000", 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155329 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194568+0000", 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013594599999999999 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20982 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.247624008999999, 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0231241369999999, 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:52.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20982 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:52.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171935+0000", 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1200000000000003e-07 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171938+0000", 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9550000000000001e-06 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171944+0000", 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171948+0000", 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8519999999999997e-06 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171977+0000", 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9539999999999998e-05 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171981+0000", 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.478e-06 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194852+0000", 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0228714160000001 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194908+0000", 2024-09-15T00:10:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5837000000000001e-05 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.195059+0000", 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015047699999999999 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:52.518 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:52.520 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:52.520 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:52.518+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:52.620 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:52.619+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:52.669 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:52.669 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:52.669 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:52.669 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:52.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:52.721 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:52.720+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:52.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:52.786 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:52.786 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:52.786 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:52.786 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:52.786 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:52.796 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:52.839 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:52.837+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:52.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:52.861 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:52.878 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:52.878 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:52.895 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:52.896 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:52.896 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:52.905 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:52.905 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:52.913 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:52.913 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:52.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:52.916 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:52.916 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:52.958 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:52.956+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:53.024 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:53.022+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:53.029 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:53.030 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:53.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:53.125 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:53.123+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:53.183 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:53.184 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:53.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:53.236 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:53.235+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:53.295 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:53.295 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:53.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:53.320 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:10:53.365 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:53.363+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:53.427 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:53.425+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:53.454 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:53.454 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:53.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:53.528 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:53.526+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:53.568 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:53.569 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:53.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:53.629 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:53.628+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:53.677 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:53.678 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:53.686 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:53.731 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:53.729+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:53.793 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:53.793 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:53.802 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:53.831 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:53.830+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:53.908 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:53.908 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:53.917 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:53.960 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:53.958+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:54.024 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:54.024 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:54.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:54.034 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:54.031+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:54.134 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:54.132+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:54.181 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:54.181 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:54.189 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:54.235 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:54.233+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:54.296 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:54.296 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:54.304 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:54.321 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:10:54.335 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:54.334+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:54.418 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:54.418 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:54.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:54.471 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:54.469+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:54.538 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:54.537+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:54.540 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:54.540 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:54.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:54.639 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:54.638+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:54.694 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:54.694 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:54.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:54.741 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:54.739+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:54.805 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:54.805 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:54.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:54.858 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:54.856+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:54.917 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:54.918 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:54.918 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:54.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:54.968 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:54.966+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:55.026 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:55.043 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:55.041+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:55.043 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:55.060 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:55.070 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:55.071 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:55.077 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:55.077 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:55.078 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:55.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:55.094 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:55.097 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:55.097 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:55.143 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:55.141+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:55.224 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.339052988000006, 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:10:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.339014358, 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:10:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:10:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.338988420999996, 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.338972353000003, 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:10:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.338831257999999, 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:10:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.337974471999999, 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:10:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:10:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.337939456000001, 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.337913853000003, 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:10:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:10:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.337739587000002, 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:10:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.337508447999994, 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:10:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:10:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.336696504000003, 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:10:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:10:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:10:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.336616763999999, 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:10:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:10:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:55.280 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.336411738999999, 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:55.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:10:55.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.336200206000001, 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:10:55.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:10:55.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.333644199000005, 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:10:55.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:55.294 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.331661092999994, 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:55.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:10:55.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.331641521999998, 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:10:55.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:10:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:10:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:10:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 53.060898672, 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:10:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:10:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:10:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.155147177000003, 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:10:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.538059736999998, 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:10:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:10:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:55.318 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:55.320 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:55.318+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:55.320 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:55.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:10:55.421 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:55.419+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:55.435 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:55.435 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:55.435 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:55.436 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:55.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:55.508 INFO:teuthology.orchestra.run.smithi079.stderr:no valid command found; 10 closest matches: 2024-09-15T00:10:55.508 INFO:teuthology.orchestra.run.smithi079.stderr:0 2024-09-15T00:10:55.508 INFO:teuthology.orchestra.run.smithi079.stderr:1 2024-09-15T00:10:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:2 2024-09-15T00:10:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:abort 2024-09-15T00:10:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:assert 2024-09-15T00:10:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:bluefs debug_inject_read_zeros 2024-09-15T00:10:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:bluefs files list 2024-09-15T00:10:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:bluefs stats 2024-09-15T00:10:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:bluestore allocator dump block 2024-09-15T00:10:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:bluestore allocator fragmentation block 2024-09-15T00:10:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:admin_socket: invalid command 2024-09-15T00:10:55.511 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:10:55.511 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-15T00:10:55.529 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:55.527+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:55.576 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:55.574+0000 7ff91860e640 -1 osd.4 88 log_to_monitors true 2024-09-15T00:10:55.593 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:55.593 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:55.593 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:55.593 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:55.593 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:55.602 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:55.622 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:55.621+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:55.723 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:55.721+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:55.751 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:55.751 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:55.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:55.751 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:55.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.751 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.751 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720782431000003, 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.752 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:10:55.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:10:55.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:10:55.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:10:55.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720762684999997, 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:10:55.757 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:10:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:10:55.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:10:55.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:10:55.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:10:55.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720742735000002, 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:10:55.763 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:10:55.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:10:55.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:10:55.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:10:55.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:10:55.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.768 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720715235, 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:10:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:10:55.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720698393999996, 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:10:55.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720682396000001, 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:10:55.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.779 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:55.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:10:55.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:10:55.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720659408000003, 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:10:55.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:10:55.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720633523999993, 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:10:55.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:55.792 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720617097000002, 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:55.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720602826000004, 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.797 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:10:55.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:10:55.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:10:55.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:10:55.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:10:55.801 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.720593508999997, 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.802 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:55.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.718924497000003, 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:10:55.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:10:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:10:55.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:10:55.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:10:55.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:10:55.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.718889434000005, 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:10:55.815 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:55.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:10:55.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:10:55.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:10:55.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:10:55.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:10:55.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:10:55.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:10:55.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:10:55.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:10:55.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:10:55.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:10:55.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:10:55.821 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.821 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.821 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.718864406999998, 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.822 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:10:55.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.718840099000005, 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:10:55.827 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:10:55.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:10:55.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:10:55.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:10:55.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:10:55.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.716623716000001, 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:10:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:10:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:10:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:10:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:10:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:10:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:10:55.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:10:55.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 85.716594344000001, 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:10:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:10:55.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.052090917000001, 2024-09-15T00:10:55.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:10:55.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:10:55.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:10:55.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.850 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5644.0:745 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.576497042, 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227015319999999, 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5644", 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1022470035", 2024-09-15T00:10:55.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 745 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171868+0000", 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-06 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171876+0000", 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.593e-06 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171890+0000", 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4347e-05 2024-09-15T00:10:55.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171897+0000", 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8449999999999997e-06 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171949+0000", 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2206000000000001e-05 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:55.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171959+0000", 2024-09-15T00:10:55.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7829999999999993e-06 2024-09-15T00:10:55.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194277+0000", 2024-09-15T00:10:55.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0223181750000001 2024-09-15T00:10:55.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194432+0000", 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155329 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194568+0000", 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013594599999999999 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20982 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.576429352000002, 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0231241369999999, 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20982 2024-09-15T00:10:55.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:55.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171935+0000", 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1200000000000003e-07 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171938+0000", 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9550000000000001e-06 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171944+0000", 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:10:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171948+0000", 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8519999999999997e-06 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171977+0000", 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9539999999999998e-05 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171981+0000", 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.478e-06 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194852+0000", 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0228714160000001 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194908+0000", 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5837000000000001e-05 2024-09-15T00:10:55.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.195059+0000", 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015047699999999999 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:55.862 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:55.864 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:55.864 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:55.862+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:55.965 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:55.963+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:56.012 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:56.012 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:56.013 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:10:56.013 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:56.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:56.065 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:56.064+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:56.127 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:56.128 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:56.128 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:56.128 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:56.128 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:56.136 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:56.166 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:56.165+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:56.244 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.216633529000006, 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:56.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.216611611000005, 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:56.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:56.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:56.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:56.254 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.216590339000007, 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:56.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:56.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:56.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:56.259 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.259 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.260 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.260 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:56.260 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.214849079000004, 2024-09-15T00:10:56.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:56.260 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:56.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:56.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:56.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:56.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:56.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:56.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:56.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:56.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.266 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.267 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.267 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:56.267 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.214827173000003, 2024-09-15T00:10:56.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:56.267 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.268 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:56.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.214805080999994, 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.214773342000001, 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.214475139000001, 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.214441433999994, 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.210139361000003, 2024-09-15T00:10:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.209814491000003, 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.209404086000006, 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.209349183, 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.209317147999997, 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.207839317999998, 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.207817602999995, 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.207796123999998, 2024-09-15T00:10:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.205783019999998, 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.205763379000004, 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 86.205716831000004, 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:56.346 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:56.347 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:10:56.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:56.348 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:56.345+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:56.351 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:10:56.348+0000 7ff90bd1e640 -1 osd.4 88 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:10:56.448 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:56.446+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:56.461 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:56.461 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:56.461 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:56.461 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:56.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:56.549 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:56.548+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:56.573 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:56.573 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:56.573 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:56.573 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:56.573 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:56.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:56.666 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:56.664+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:56.727 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:56.728 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:10:56.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:10:56.728 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:10:56.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:56.729 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.812267144000003, 2024-09-15T00:10:56.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:10:56.729 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.730 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.730 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.730 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.730 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.730 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.731 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:10:56.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.731 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:56.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:56.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:10:56.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:10:56.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:10:56.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:10:56.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:10:56.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:10:56.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:10:56.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:10:56.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:10:56.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:10:56.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:10:56.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:10:56.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:10:56.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:10:56.740 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.740 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.740 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.741 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.741 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:56.741 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.812246838, 2024-09-15T00:10:56.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:10:56.741 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.741 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.742 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.742 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:10:56.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.743 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:56.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:10:56.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:10:56.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:10:56.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:10:56.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:10:56.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:10:56.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:10:56.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:10:56.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:10:56.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:10:56.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:10:56.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:10:56.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:10:56.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.812224897999997, 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:10:56.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.810367126999999, 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:10:56.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.810337447999999, 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:10:56.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:10:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:10:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:10:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:10:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.810314355000003, 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:10:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.810289181999998, 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.810268540999999, 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:10:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:10:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:10:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:10:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:10:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:10:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:10:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.810248000000001, 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:10:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.810224517999998, 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.808360862000001, 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:10:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:10:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:10:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.808331093, 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:10:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:10:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.808308914000001, 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:10:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:10:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:10:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:10:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.808281663999999, 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:10:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:10:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:10:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:10:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.808255070000001, 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:10:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:10:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:10:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:10:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:10:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:10:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.808235736999997, 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:10:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.806187328, 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:10:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.806172095000001, 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:10:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:10:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:10:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.040895575999997, 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:10:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.994804905999999, 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:10:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:10:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:10:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:10:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:10:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:10:56.846 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:56.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:56.847 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:56.845+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:56.947 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:56.946+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:56.991 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:56.991 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:56.991 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:56.991 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:56.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:57.048 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:57.046+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:57.099 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:10:57.142 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:57.142 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:57.142 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:10:57.142 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:10:57.142 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:57.149 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:57.147+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:57.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:57.207 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:57.224 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:57.225 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:57.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:57.241 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:57.249 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:57.248+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:57.258 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:57.258 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:57.258 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-15T00:10:57.275 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:57.275 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:57.278 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:57.278 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:57.295 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:57.295 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:57.303 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:10:57.351 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:57.349+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:57.418 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:57.418 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:57.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:57.451 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:57.450+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:57.535 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:57.535 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:57.544 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:57.588 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:57.586+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:57.652 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:57.652 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:57.653 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:57.652+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:57.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:57.754 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:57.752+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:57.808 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:57.808 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:57.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:57.861 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:57.859+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:57.924 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:57.924 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:57.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:57.974 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:57.972+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:58.057 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:58.055+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:58.081 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:58.081 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:58.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:58.158 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:58.156+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:58.199 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:58.199 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:58.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:58.258 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:58.257+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:10:58.310 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:58.311 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:58.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:10:58.361 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:58.359+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:58.461 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:58.459+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:58.494 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:58.494 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:58.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:10:58.561 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:58.559+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:58.662 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:58.661+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:58.669 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:58.669 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:58.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:10:58.764 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:58.763+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:58.779 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:58.779 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:58.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:10:58.865 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:58.864+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:58.891 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:58.891 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:58.900 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:10:58.966 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:58.965+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:59.009 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:59.009 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:59.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:10:59.067 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:59.066+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:59.125 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:59.125 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:59.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:10:59.177 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:10:59.175+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:10:59.240 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:59.241 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:59.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:10:59.269 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:59.268+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:59.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:10:59.359 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:10:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:10:59.400 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:59.397+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:59.403 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:10:59.403 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:10:59.413 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:10:59.418 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:10:59.418 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:10:59.436 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:10:59.436 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:10:59.453 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:10:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:10:59.471 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:10:59.471 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-15T00:10:59.472 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:59.470+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:10:59.492 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:10:59.492 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:10:59.492 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:10:59.531 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:59.531 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:10:59.531 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:10:59.532 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:10:59.532 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:59.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:10:59.582 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:59.580+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:59.674 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:59.672+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:59.691 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.663540695999998, 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.692 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.693 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.693 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:10:59.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.693 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:10:59.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.663518777999997, 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.700 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:59.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:10:59.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.704 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.663497505999999, 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:10:59.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.661756245999996, 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.709 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:10:59.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.661734339999995, 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:10:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:10:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.661712248000001, 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.661680509000007, 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:10:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.661382305999993, 2024-09-15T00:10:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:10:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:10:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:10:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.661348601, 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:10:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.657046527999995, 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:10:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.656721657999995, 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.656311252999998, 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:10:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:10:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.656256350000007, 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:10:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.656224315000003, 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:10:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:10:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:10:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.654746485000004, 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.654724770000001, 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:10:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:10:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.654703291000004, 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:10:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.652690187000005, 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:10:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.652670545999996, 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:10:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 89.652623997999996, 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:10:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:10:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:10:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:10:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:10:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:10:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:10:59.787 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:10:59.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:10:59.789 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:10:59.787+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:10:59.889 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:10:59.888+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:10:59.891 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:10:59.891 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:10:59.891 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:10:59.891 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:10:59.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:10:59.990 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:10:59.989+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:00.044 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:00.044 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:00.044 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:00.044 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:00.045 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:00.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:00.094 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:00.092+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:00.192 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:00.191+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.282563164999999, 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:11:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:11:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.282542859000003, 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:11:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.282520919, 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:11:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:11:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.280663148000002, 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:11:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:11:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.280633469000001, 2024-09-15T00:11:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:11:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.280610375999998, 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:11:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.280585203000001, 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:11:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:11:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.280564562000002, 2024-09-15T00:11:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:11:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:11:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.280544020999997, 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.280520539000001, 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:11:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.278656882999996, 2024-09-15T00:11:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:11:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:11:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.278627114000003, 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:11:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.278604934999997, 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:11:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.278577685000002, 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:11:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:11:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.278551090999997, 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.278531758, 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:11:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.276483349000003, 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:11:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:11:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.276468115999997, 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:11:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.511191597, 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:11:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:11:00.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.465100927000002, 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:00.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:11:00.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:11:00.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.292 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:00.293 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:00.292+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:00.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:00.366 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:11:00.424 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:00.422+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:00.446 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:00.446 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:00.446 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:11:00.446 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:00.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:00.498 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:00.497+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:00.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:00.560 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:00.561 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:00.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:00.595 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:00.594+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:00.619 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:25995 3.3 3.5a749323 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.19141460299999999, 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.19143666100000001, 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 25995 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427092+0000", 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.48e-06 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427100+0000", 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.072e-06 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427113+0000", 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2845e-05 2024-09-15T00:11:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427124+0000", 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0324e-05 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427177+0000", 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3514999999999998e-05 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-15T00:11:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427184+0000", 2024-09-15T00:11:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1910000000000003e-06 2024-09-15T00:11:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:11:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-15T00:11:00.624 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:00.676 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.791238264, 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:11:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:11:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.79119963399999, 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:11:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:11:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.791173697, 2024-09-15T00:11:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:11:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.791157629, 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:11:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.79101653399999, 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:11:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.79015974799999, 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:11:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.790124732, 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:11:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:11:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:11:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:11:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:11:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.790099129, 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:11:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.789924863, 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:11:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.789693724, 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.78888178, 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:11:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.78880203999999, 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:11:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.78859701499999, 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:11:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.788385482, 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:11:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:11:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.785829475, 2024-09-15T00:11:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:11:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.783846369, 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:11:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.78382679800001, 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:11:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 58.513083948000002, 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:11:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:11:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:11:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 45.607332452999998, 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:11:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.990245012999999, 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:11:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:11:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:11:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.775 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:00.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:00.777 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:00.775+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:00.878 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:00.876+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:00.881 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:00.881 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:00.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.881 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:25995 3.3 3.5a749323 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.45318682599999999, 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.45320005200000002, 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 25995 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:00.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:00.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427092+0000", 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.48e-06 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427100+0000", 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.072e-06 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427113+0000", 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2845e-05 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427124+0000", 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0324e-05 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427177+0000", 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3514999999999998e-05 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:00.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-15T00:11:00.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427184+0000", 2024-09-15T00:11:00.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1910000000000003e-06 2024-09-15T00:11:00.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.886 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:00.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:00.886 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:11:00.886 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-15T00:11:00.886 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:00.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:00.978 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:00.977+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:01.033 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:01.033 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:01.033 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:01.033 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:01.033 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:01.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:01.080 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:01.078+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:01.181 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:01.179+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:01.186 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1893 3.1e e119/118 3:7c90e45f:::rbd_data.11291f9c3154.000000000000003a:1d v 119'463, mlcod=119'459)", 2024-09-15T00:11:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.755268606, 2024-09-15T00:11:01.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005948208, 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.37e-07 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428523+0000", 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2139999999999996e-06 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428532+0000", 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7229999999999999e-06 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428539+0000", 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3289999999999999e-06 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428578+0000", 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8792999999999999e-05 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428595+0000", 2024-09-15T00:11:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7669e-05 2024-09-15T00:11:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.434405+0000", 2024-09-15T00:11:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058096780000000004 2024-09-15T00:11:01.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.434462+0000", 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7065000000000003e-05 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3933 3.15 e119/118 3:ad39a9e7:::rbd_data.11291d0c7d7c.0000000000000000:16 v 119'404, mlcod=115'401)", 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7539442869999999, 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075329059999999998, 2024-09-15T00:11:01.191 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429839+0000", 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5799999999999999e-07 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429844+0000", 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7649999999999999e-06 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429855+0000", 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0644e-05 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429862+0000", 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.477e-06 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429903+0000", 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1693999999999998e-05 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429931+0000", 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7699e-05 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.437329+0000", 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073982270000000003 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.437371+0000", 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1742000000000001e-05 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.195 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1895 3.1e e119/118 3:7d7cb6ca:::rbd_data.11294a896ed0.0000000000000010:22 v 119'467, mlcod=119'463)", 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7434462509999999, 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007846713, 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:01.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440341+0000", 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7350000000000004e-06 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440349+0000", 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6299999999999998e-06 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440357+0000", 2024-09-15T00:11:01.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1710000000000002e-06 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440413+0000", 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5980000000000003e-05 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440432+0000", 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8847000000000001e-05 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.448132+0000", 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007700386 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.448183+0000", 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0559000000000002e-05 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3934 3.15 e119/118 3:a9ed06fd:::rbd_data.112986cdeff1.0000000000000014:1b v 119'406, mlcod=116'402)", 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7429633579999999, 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010330291, 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440820+0000", 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3099999999999997e-07 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440825+0000", 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.186e-06 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440836+0000", 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.092e-05 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440842+0000", 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3890000000000002e-06 2024-09-15T00:11:01.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440903+0000", 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1393e-05 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440921+0000", 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7975e-05 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451109+0000", 2024-09-15T00:11:01.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010187391000000001 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451149+0000", 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0405999999999999e-05 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(osd.4.0:11 3.1a e119/118 ondisk, result = 0)", 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.441332+0000", 2024-09-15T00:11:01.205 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.742450367, 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004360269, 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441332+0000", 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441332+0000", 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441333+0000", 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3900000000000004e-07 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441340+0000", 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5050000000000001e-06 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441350+0000", 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441354+0000", 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.51e-06 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.445666+0000", 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043116070000000003 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.445675+0000", 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5799999999999998e-06 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.445692+0000", 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6994999999999999e-05 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1897 3.1e e119/118 3:7a2239c1:::rbd_data.1129466cbf24.000000000000000a:27 v 119'471, mlcod=119'465)", 2024-09-15T00:11:01.209 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7327991269999998, 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005507311, 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450984+0000", 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450987+0000", 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9409999999999999e-06 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450992+0000", 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2789999999999998e-06 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450995+0000", 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0740000000000001e-06 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451734+0000", 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073883599999999998 2024-09-15T00:11:01.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451756+0000", 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2158999999999999e-05 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.456450+0000", 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004693666 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.456491+0000", 2024-09-15T00:11:01.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0976000000000002e-05 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3935 3.15 e119/118 3:acdb5df2:::rbd_data.11298669a7cc.0000000000000006:21 v 119'408, mlcod=119'404)", 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.455355+0000", 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.728427656, 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039587279999999999, 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.214 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.455355+0000", 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.455355+0000", 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.455355+0000", 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9999999999999997e-07 2024-09-15T00:11:01.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.455361+0000", 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4750000000000001e-06 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.455371+0000", 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0132999999999999e-05 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.455380+0000", 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3400000000000004e-06 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.455458+0000", 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7564000000000005e-05 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.455480+0000", 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1713999999999999e-05 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.459269+0000", 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037898889999999998 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.459313+0000", 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3912999999999998e-05 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.218 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3936 3.15 e119/118 3:a9d1a5ec:::rbd_data.1129696a4676.0000000000000006:24 v 119'410, mlcod=119'406)", 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.722151963, 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006441413, 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461631+0000", 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4899999999999995e-07 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461636+0000", 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6659999999999999e-06 2024-09-15T00:11:01.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461644+0000", 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5269999999999998e-06 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461653+0000", 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5620000000000004e-06 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461705+0000", 2024-09-15T00:11:01.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2045000000000001e-05 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461724+0000", 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.893e-05 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.468031+0000", 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006307773 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.468072+0000", 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0361000000000002e-05 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3937 3.15 e119/118 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:27 v 119'412, mlcod=119'408)", 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.709581773, 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077903419999999996, 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.223 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474207+0000", 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0820000000000004e-06 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474218+0000", 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.111e-05 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474225+0000", 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7830000000000001e-06 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474267+0000", 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1644999999999999e-05 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474288+0000", 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0693999999999999e-05 2024-09-15T00:11:01.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.481944+0000", 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076562280000000002 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.481991+0000", 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7083000000000003e-05 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5280 3.e e119/118 3:74824508:::rbd_data.1129eb1f5358.000000000000000d:18 v 119'283, mlcod=110'280)", 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.485884+0000", 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.697898736, 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044121849999999999, 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485884+0000", 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485884+0000", 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485884+0000", 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9800000000000003e-07 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485891+0000", 2024-09-15T00:11:01.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0929999999999997e-06 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485906+0000", 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4218e-05 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485921+0000", 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5037e-05 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485991+0000", 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0288000000000002e-05 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486015+0000", 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4539e-05 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.490254+0000", 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042382590000000003 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.490296+0000", 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2052999999999999e-05 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5281 3.e e119/118 3:771a58a1:::rbd_data.1129eb1f5358.0000000000000007:18 v 119'285, mlcod=110'280)", 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.6972646999999998, 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064605180000000002, 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486523+0000", 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486531+0000", 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2570000000000001e-06 2024-09-15T00:11:01.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486535+0000", 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4170000000000002e-06 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486578+0000", 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099000000000002e-05 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486599+0000", 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0735999999999999e-05 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.492941+0000", 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063422289999999996 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.492978+0000", 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7045000000000002e-05 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5282 3.e e119/118 3:701c32a5:::rbd_data.112929d537d9.00000000000001c7:1e v 119'287, mlcod=119'283)", 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.687220741, 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046705490000000004, 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:01.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:01.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2099999999999996e-07 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496568+0000", 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0170000000000002e-06 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496577+0000", 2024-09-15T00:11:01.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8939999999999999e-06 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496586+0000", 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5080000000000005e-06 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496625+0000", 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9140999999999997e-05 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496645+0000", 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0248999999999999e-05 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.501202+0000", 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045572820000000002 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.501232+0000", 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9836999999999999e-05 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.241 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5283 3.e e119/118 3:7642e0f1:::rbd_data.112929d537d9.000000000000002c:1e v 119'289, mlcod=119'283)", 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.6865621200000001, 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064221740000000001, 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497221+0000", 2024-09-15T00:11:01.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7299999999999995e-07 2024-09-15T00:11:01.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497226+0000", 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0239999999999997e-06 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497238+0000", 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1746000000000001e-05 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497244+0000", 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6640000000000001e-06 2024-09-15T00:11:01.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497287+0000", 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3130999999999998e-05 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497307+0000", 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9740000000000001e-05 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.503604+0000", 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062964780000000003 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.503642+0000", 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8717999999999998e-05 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.247 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3942 3.14 e119/118 3:2c03414e:::rbd_data.11299af7e2c0.0000000000000001:15 v 119'546, mlcod=112'543)", 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.591897+0000", 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.591885225, 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040778009999999998, 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.591897+0000", 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.591897+0000", 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.591898+0000", 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7700000000000004e-07 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.591903+0000", 2024-09-15T00:11:01.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3260000000000003e-06 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.591912+0000", 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2129999999999997e-06 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.591918+0000", 2024-09-15T00:11:01.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8989999999999998e-06 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.591963+0000", 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4468000000000002e-05 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.591974+0000", 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.153e-05 2024-09-15T00:11:01.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.595935+0000", 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039604489999999996 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.595975+0000", 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0339000000000001e-05 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3943 3.14 e119/118 3:2a8444a6:::rbd_data.11291d0c7d7c.0000000000000003:16 v 119'548, mlcod=116'544)", 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.586969775, 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064816789999999997, 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:01.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6300000000000005e-07 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596819+0000", 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596827+0000", 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1200000000000002e-06 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596832+0000", 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5179999999999998e-06 2024-09-15T00:11:01.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596870+0000", 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.803e-05 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596891+0000", 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1746000000000002e-05 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.603253+0000", 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006361663 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.603294+0000", 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1220000000000002e-05 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3944 3.14 e119/118 3:2e97003a:::rbd_data.1129eb1f5358.000000000000023e:18 v 119'550, mlcod=119'546)", 2024-09-15T00:11:01.257 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.5788415279999999, 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048495739999999997, 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604946+0000", 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:11:01.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604953+0000", 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4669999999999998e-06 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604957+0000", 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604993+0000", 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5578999999999999e-05 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.605007+0000", 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3599e-05 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.609752+0000", 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004745768 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.609790+0000", 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7972999999999997e-05 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3945 3.14 e119/118 3:28044945:::rbd_data.11299a2b31f2.0000000000000004:1a v 119'552, mlcod=119'548)", 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.611085+0000", 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.572697196, 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035591239999999999, 2024-09-15T00:11:01.262 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.611085+0000", 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.611085+0000", 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.611086+0000", 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8999999999999996e-07 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.611091+0000", 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1490000000000002e-06 2024-09-15T00:11:01.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.611099+0000", 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4419999999999998e-06 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.611104+0000", 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.527e-06 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.611141+0000", 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6733999999999998e-05 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.611161+0000", 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0806e-05 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.614600+0000", 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003438188 2024-09-15T00:11:01.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.614644+0000", 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4688000000000003e-05 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3956 3.11 e119/118 3:8f5f1221:::rbd_data.1129466cbf24.0000000000000009:27 v 119'1188, mlcod=119'1182)", 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.644540+0000", 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.539242073, 2024-09-15T00:11:01.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033549470000000001, 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.644540+0000", 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.644540+0000", 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.644541+0000", 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6300000000000005e-07 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.644545+0000", 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4410000000000003e-06 2024-09-15T00:11:01.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.644555+0000", 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8539999999999992e-06 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.644560+0000", 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4920000000000004e-06 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.644593+0000", 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2923000000000003e-05 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.644605+0000", 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2181000000000001e-05 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.647864+0000", 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032588339999999999 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.647895+0000", 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1659000000000003e-05 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1899 3.1e e121/118 3:7d005a47:::rbd_data.112951ba645b.000000000000000d:13 v 121'473, mlcod=119'469)", 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.81441122399999999, 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004557752, 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.272 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369376+0000", 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5480000000000003e-06 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369392+0000", 2024-09-15T00:11:01.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5862000000000002e-05 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369400+0000", 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0290000000000005e-06 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369457+0000", 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7163999999999997e-05 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369478+0000", 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1461999999999999e-05 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.373883+0000", 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044043550000000004 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.373929+0000", 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6071e-05 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3963 3.11 e121/118 3:887e724c:::rbd_data.112951ba645b.0000000000000028:13 v 121'1190, mlcod=119'1186)", 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:00.370446+0000", 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.81333607900000005, 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039068560000000002, 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.370446+0000", 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.370446+0000", 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.370447+0000", 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4799999999999997e-07 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.370453+0000", 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.694e-06 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.370461+0000", 2024-09-15T00:11:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8760000000000005e-06 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.370469+0000", 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.371231+0000", 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076256499999999999 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.371250+0000", 2024-09-15T00:11:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9320000000000001e-05 2024-09-15T00:11:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.374290+0000", 2024-09-15T00:11:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030392359999999998 2024-09-15T00:11:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.374353+0000", 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3406999999999998e-05 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:01.282 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:01.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:01.285 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:01.281+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:01.384 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:01.382+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:01.407 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:01.407 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:11:01.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:01.407 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:26017 3.0 3:07efe3c5:::rbd_object_map.112951ba645b:head [call lock.lock in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-15T00:11:01.407 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:11:01.406118+0000", 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.00016877400000000001, 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000190886, 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 26017 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:01.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:11:01.406118+0000", 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:11:01.406118+0000", 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:11:01.406120+0000", 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.249e-06 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:11:01.406126+0000", 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4609999999999996e-06 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:11:01.406141+0000", 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4538999999999999e-05 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:11:01.406147+0000", 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9889999999999997e-06 2024-09-15T00:11:01.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:11:01.406211+0000", 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4808999999999996e-05 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:11:01.406279+0000", 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7657999999999995e-05 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T00:11:01.412 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 1 2024-09-15T00:11:01.412 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:01.419 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:01.485 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:01.483+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:01.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:01.529 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:01.529 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:01.529 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:11:01.529 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:11:01.529 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:01.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:01.586 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:01.585+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:01.602 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:11:01.620 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:11:01.620 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:01.638 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:11:01.638 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:01.655 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:11:01.655 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:01.672 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:11:01.673 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:01.687 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:01.685+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:01.689 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:01.689 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:01.689 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:11:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:01.693 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:11:01.693 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:11:01.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:01.788 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:01.787+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:01.847 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:01.847 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:01.856 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:01.889 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:01.888+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:01.963 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:01.963 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:01.971 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:01.989 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:01.988+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:02.079 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:02.079 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:02.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:02.091 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:02.089+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:02.190 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:02.189+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:02.195 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:02.195 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:02.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:02.291 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:02.290+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:02.346 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:02.347 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:02.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:02.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:11:02.397 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:02.395+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:02.493 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:02.492+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:02.510 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:02.510 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:02.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:02.603 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:02.602+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:02.664 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:02.664 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:02.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:02.714 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:02.712+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:02.796 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:02.795+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:02.815 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:02.816 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:02.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:02.907 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:02.906+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:02.967 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:02.967 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:02.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:03.017 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:03.015+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:03.099 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:03.098+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:03.118 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:03.118 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:03.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:03.200 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:03.199+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:03.229 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:03.229 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:03.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:03.321 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:03.320+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:03.380 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:03.381 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:03.385 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:11:03.389 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:03.431 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:03.429+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:03.503 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:03.501+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:03.506 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:03.506 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:03.516 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:03.604 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:03.602+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:03.672 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:03.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:03.713 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:03.711+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:03.805 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:03.804+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:03.819 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:11:03.822 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:03.822 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:03.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:03.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:11:03.837 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:03.856 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:11:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:03.873 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:11:03.873 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:03.890 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:11:03.890 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:03.906 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:03.904+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:03.907 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:11:03.907 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:03.910 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:11:03.910 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:11:03.941 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:03.941 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:03.941 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:11:03.941 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:03.950 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:04.034 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:04.032+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:04.097 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:04.098 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:04.098 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:11:04.098 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:11:04.098 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:04.106 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:04.107 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:04.106+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:04.208 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:04.207+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:04.255 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:04.255 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:11:04.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:11:04.255 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:11:04.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.255 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.255 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:04.255 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.226962998999994, 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:11:04.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:11:04.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.226941081000007, 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.260 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:04.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.226919808999995, 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:11:04.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.225178549000006, 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:11:04.269 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:11:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.273 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.225156643000005, 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.274 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:11:04.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.278 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.279 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.279 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:11:04.279 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.225134550999996, 2024-09-15T00:11:04.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:11:04.279 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:11:04.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:11:04.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:11:04.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:11:04.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.225102812000003, 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:11:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.224804609000003, 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:11:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.224770903999996, 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:11:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.220468831000005, 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.299 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.220143961000005, 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:11:04.303 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:11:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:11:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.219733555999994, 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:11:04.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:11:04.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.219678653000003, 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:11:04.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.219646617999999, 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:11:04.317 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:11:04.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.218168788, 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.218147072999997, 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.218125594, 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:11:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:11:04.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.21611249, 2024-09-15T00:11:04.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:11:04.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:11:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.216092849000006, 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.341 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:11:04.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.345 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 94.216046301000006, 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.346 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:04.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:04.350 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:04.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:04.352 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:04.350+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:04.452 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:04.450+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:04.472 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:26492 3.1f 3:faeee0d0:::rbd_header.1129ebdcc4e7:head [watch watch cookie 140653334294896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-15T00:11:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:04.470536+0000", 2024-09-15T00:11:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00073498200000000002, 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000751507, 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 26492 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:04.470536+0000", 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:04.470536+0000", 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:04.470536+0000", 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5799999999999998e-07 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:04.470539+0000", 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.044e-06 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:04.470549+0000", 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7170000000000003e-06 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:04.470555+0000", 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5809999999999996e-06 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:04.470586+0000", 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1025000000000002e-05 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:04.470630+0000", 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4471000000000002e-05 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:04.470715+0000", 2024-09-15T00:11:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4550999999999997e-05 2024-09-15T00:11:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:11:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-15T00:11:04.478 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:04.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:04.553 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:04.551+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:04.627 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:04.628 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:04.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:04.680 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:04.678+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:04.740 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.825669853999997, 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:11:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.825649548000001, 2024-09-15T00:11:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:11:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.825627607999998, 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:11:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.823769837, 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.823740158, 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:11:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.823717064999997, 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.823691891999999, 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:11:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.823671251, 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:11:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.823650710000003, 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:11:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.823627227999999, 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:11:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.821763572000002, 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:11:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:11:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.821733803000001, 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:11:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:11:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.821711624000002, 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.821684374, 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.821657780000002, 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:11:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.821638446999998, 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:11:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:11:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:11:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.819590038000001, 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:11:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:11:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.819574805000002, 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:11:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.054298285999998, 2024-09-15T00:11:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:11:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.008207616, 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:11:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:11:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:04.837 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:04.837 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:04.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:04.839 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:04.836+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:04.938 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:04.937+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:04.981 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:11:04.981 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:04.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:05.075 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:05.073+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:05.133 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:05.140 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:05.139+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:05.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:05.242 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:05.240+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:05.287 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:05.287 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:05.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:05.287 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:05.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.287 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.401633511, 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.288 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.40159488099999, 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:11:05.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:05.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:11:05.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:11:05.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:11:05.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:11:05.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.401568944, 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.299 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:11:05.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:05.303 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.401552876, 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:05.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:11:05.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:11:05.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:11:05.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:11:05.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:11:05.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.40141178099999, 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.309 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.40055499499999, 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:11:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:11:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.40051997899999, 2024-09-15T00:11:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:11:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:11:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:11:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:11:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:11:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:11:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:11:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:11:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.400494376, 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:11:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:11:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:11:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.40032011, 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:11:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:11:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.400088971, 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:11:05.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:11:05.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:11:05.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:11:05.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:05.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.399277027, 2024-09-15T00:11:05.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:05.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:05.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:11:05.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:11:05.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:11:05.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:11:05.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:11:05.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:11:05.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:11:05.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:11:05.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:11:05.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:11:05.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:11:05.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:11:05.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:11:05.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:11:05.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.348 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.348 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:05.348 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.39919728700001, 2024-09-15T00:11:05.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:11:05.348 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.348 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.349 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:11:05.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.349 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:05.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:05.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:11:05.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:11:05.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:11:05.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:11:05.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:11:05.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:11:05.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:11:05.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.39899226199999, 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:11:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:11:05.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.39878072899999, 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:11:05.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:11:05.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.396224722, 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.364 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.394241616, 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.369 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:05.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.373 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.39422204500001, 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.378 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 63.123479195000002, 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:11:05.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.217727699999998, 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:11:05.383 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:11:05.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:11:05.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:11:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:11:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.600640259999999, 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.389 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:11:05.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:11:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:11:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:11:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.394 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.394 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:05.395 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:05.393+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:05.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:05.406 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:11:05.496 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:05.494+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:05.548 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:11:05.549 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:05.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:05.597 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:05.595+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:05.629 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [0] live_osds: [1, 2, 3, 5, 4] 2024-09-15T00:11:05.630 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-15T00:11:05.630 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T00:11:05.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T00:11:05.632 INFO:tasks.ceph.osd.0:Started 2024-09-15T00:11:05.632 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:05.699 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:05.696+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:05.699 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:05.700 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:05.700 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:05.700 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:05.700 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:05.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:05.735 INFO:teuthology.orchestra.run.smithi027.stderr:no valid command found; 10 closest matches: 2024-09-15T00:11:05.735 INFO:teuthology.orchestra.run.smithi027.stderr:0 2024-09-15T00:11:05.735 INFO:teuthology.orchestra.run.smithi027.stderr:1 2024-09-15T00:11:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2 2024-09-15T00:11:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:abort 2024-09-15T00:11:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:assert 2024-09-15T00:11:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:config diff 2024-09-15T00:11:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:config diff get 2024-09-15T00:11:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:config get 2024-09-15T00:11:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:config help [] 2024-09-15T00:11:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:config set ... 2024-09-15T00:11:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:admin_socket: invalid command 2024-09-15T00:11:05.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:11:05.738 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T00:11:05.799 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:05.798+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:05.812 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1893 3.1e e119/118 3:7c90e45f:::rbd_data.11291f9c3154.000000000000003a:1d v 119'463, mlcod=119'459)", 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3818959900000003, 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005948208, 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.37e-07 2024-09-15T00:11:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428523+0000", 2024-09-15T00:11:05.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2139999999999996e-06 2024-09-15T00:11:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428532+0000", 2024-09-15T00:11:05.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7229999999999999e-06 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428539+0000", 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3289999999999999e-06 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428578+0000", 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8792999999999999e-05 2024-09-15T00:11:05.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428595+0000", 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7669e-05 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.434405+0000", 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058096780000000004 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.434462+0000", 2024-09-15T00:11:05.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7065000000000003e-05 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3933 3.15 e119/118 3:ad39a9e7:::rbd_data.11291d0c7d7c.0000000000000000:16 v 119'404, mlcod=115'401)", 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3805716710000002, 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075329059999999998, 2024-09-15T00:11:05.819 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429839+0000", 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5799999999999999e-07 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429844+0000", 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7649999999999999e-06 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429855+0000", 2024-09-15T00:11:05.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0644e-05 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429862+0000", 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.477e-06 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429903+0000", 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1693999999999998e-05 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429931+0000", 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7699e-05 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.437329+0000", 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073982270000000003 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.437371+0000", 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1742000000000001e-05 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1895 3.1e e119/118 3:7d7cb6ca:::rbd_data.11294a896ed0.0000000000000010:22 v 119'467, mlcod=119'463)", 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3700736349999998, 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007846713, 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440341+0000", 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7350000000000004e-06 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440349+0000", 2024-09-15T00:11:05.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6299999999999998e-06 2024-09-15T00:11:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440357+0000", 2024-09-15T00:11:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1710000000000002e-06 2024-09-15T00:11:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440413+0000", 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5980000000000003e-05 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440432+0000", 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8847000000000001e-05 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.448132+0000", 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007700386 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.448183+0000", 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0559000000000002e-05 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3934 3.15 e119/118 3:a9ed06fd:::rbd_data.112986cdeff1.0000000000000014:1b v 119'406, mlcod=116'402)", 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3695907419999998, 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010330291, 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440820+0000", 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3099999999999997e-07 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440825+0000", 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.186e-06 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440836+0000", 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.092e-05 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440842+0000", 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3890000000000002e-06 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440903+0000", 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1393e-05 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440921+0000", 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7975e-05 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451109+0000", 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010187391000000001 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451149+0000", 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0405999999999999e-05 2024-09-15T00:11:05.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(osd.4.0:11 3.1a e119/118 ondisk, result = 0)", 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.441332+0000", 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3690777509999998, 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004360269, 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:05.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441332+0000", 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441332+0000", 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441333+0000", 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3900000000000004e-07 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441340+0000", 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5050000000000001e-06 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441350+0000", 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.441354+0000", 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.51e-06 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.445666+0000", 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043116070000000003 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.445675+0000", 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5799999999999998e-06 2024-09-15T00:11:05.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.445692+0000", 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6994999999999999e-05 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1897 3.1e e119/118 3:7a2239c1:::rbd_data.1129466cbf24.000000000000000a:27 v 119'471, mlcod=119'465)", 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3594265109999997, 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005507311, 2024-09-15T00:11:05.838 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450984+0000", 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450987+0000", 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9409999999999999e-06 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450992+0000", 2024-09-15T00:11:05.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2789999999999998e-06 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450995+0000", 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0740000000000001e-06 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451734+0000", 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073883599999999998 2024-09-15T00:11:05.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451756+0000", 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2158999999999999e-05 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.456450+0000", 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004693666 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.456491+0000", 2024-09-15T00:11:05.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0976000000000002e-05 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3936 3.15 e119/118 3:a9d1a5ec:::rbd_data.1129696a4676.0000000000000006:24 v 119'410, mlcod=119'406)", 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3487793469999998, 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006441413, 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461631+0000", 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4899999999999995e-07 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461636+0000", 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6659999999999999e-06 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461644+0000", 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5269999999999998e-06 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461653+0000", 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5620000000000004e-06 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461705+0000", 2024-09-15T00:11:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2045000000000001e-05 2024-09-15T00:11:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461724+0000", 2024-09-15T00:11:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.893e-05 2024-09-15T00:11:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.468031+0000", 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006307773 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.468072+0000", 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0361000000000002e-05 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3937 3.15 e119/118 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:27 v 119'412, mlcod=119'408)", 2024-09-15T00:11:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:05.849 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3362091569999999, 2024-09-15T00:11:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077903419999999996, 2024-09-15T00:11:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:11:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474207+0000", 2024-09-15T00:11:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0820000000000004e-06 2024-09-15T00:11:05.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474218+0000", 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.111e-05 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474225+0000", 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7830000000000001e-06 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474267+0000", 2024-09-15T00:11:05.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1644999999999999e-05 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474288+0000", 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0693999999999999e-05 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.481944+0000", 2024-09-15T00:11:05.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076562280000000002 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.481991+0000", 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7083000000000003e-05 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.855 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5280 3.e e119/118 3:74824508:::rbd_data.1129eb1f5358.000000000000000d:18 v 119'283, mlcod=110'280)", 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.485884+0000", 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3245261199999998, 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044121849999999999, 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485884+0000", 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485884+0000", 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485884+0000", 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9800000000000003e-07 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485891+0000", 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0929999999999997e-06 2024-09-15T00:11:05.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485906+0000", 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4218e-05 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485921+0000", 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5037e-05 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.485991+0000", 2024-09-15T00:11:05.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0288000000000002e-05 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486015+0000", 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4539e-05 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.490254+0000", 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042382590000000003 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.490296+0000", 2024-09-15T00:11:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2052999999999999e-05 2024-09-15T00:11:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5281 3.e e119/118 3:771a58a1:::rbd_data.1129eb1f5358.0000000000000007:18 v 119'285, mlcod=110'280)", 2024-09-15T00:11:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:05.860 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3238920839999997, 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064605180000000002, 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:11:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486523+0000", 2024-09-15T00:11:05.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:11:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486531+0000", 2024-09-15T00:11:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2570000000000001e-06 2024-09-15T00:11:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486535+0000", 2024-09-15T00:11:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4170000000000002e-06 2024-09-15T00:11:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486578+0000", 2024-09-15T00:11:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099000000000002e-05 2024-09-15T00:11:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486599+0000", 2024-09-15T00:11:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0735999999999999e-05 2024-09-15T00:11:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.492941+0000", 2024-09-15T00:11:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063422289999999996 2024-09-15T00:11:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.492978+0000", 2024-09-15T00:11:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7045000000000002e-05 2024-09-15T00:11:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5282 3.e e119/118 3:701c32a5:::rbd_data.112929d537d9.00000000000001c7:1e v 119'287, mlcod=119'283)", 2024-09-15T00:11:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3138481249999998, 2024-09-15T00:11:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046705490000000004, 2024-09-15T00:11:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2099999999999996e-07 2024-09-15T00:11:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496568+0000", 2024-09-15T00:11:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0170000000000002e-06 2024-09-15T00:11:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496577+0000", 2024-09-15T00:11:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8939999999999999e-06 2024-09-15T00:11:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496586+0000", 2024-09-15T00:11:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5080000000000005e-06 2024-09-15T00:11:05.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496625+0000", 2024-09-15T00:11:05.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9140999999999997e-05 2024-09-15T00:11:05.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496645+0000", 2024-09-15T00:11:05.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0248999999999999e-05 2024-09-15T00:11:05.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.501202+0000", 2024-09-15T00:11:05.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045572820000000002 2024-09-15T00:11:05.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.501232+0000", 2024-09-15T00:11:05.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9836999999999999e-05 2024-09-15T00:11:05.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.881 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5283 3.e e119/118 3:7642e0f1:::rbd_data.112929d537d9.000000000000002c:1e v 119'289, mlcod=119'283)", 2024-09-15T00:11:05.881 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:05.881 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.3131895040000003, 2024-09-15T00:11:05.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064221740000000001, 2024-09-15T00:11:05.882 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.882 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.882 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.882 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:05.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:05.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497221+0000", 2024-09-15T00:11:05.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7299999999999995e-07 2024-09-15T00:11:05.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497226+0000", 2024-09-15T00:11:05.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0239999999999997e-06 2024-09-15T00:11:05.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497238+0000", 2024-09-15T00:11:05.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1746000000000001e-05 2024-09-15T00:11:05.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497244+0000", 2024-09-15T00:11:05.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6640000000000001e-06 2024-09-15T00:11:05.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497287+0000", 2024-09-15T00:11:05.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3130999999999998e-05 2024-09-15T00:11:05.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497307+0000", 2024-09-15T00:11:05.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9740000000000001e-05 2024-09-15T00:11:05.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.503604+0000", 2024-09-15T00:11:05.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062964780000000003 2024-09-15T00:11:05.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.503642+0000", 2024-09-15T00:11:05.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8717999999999998e-05 2024-09-15T00:11:05.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.891 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3943 3.14 e119/118 3:2a8444a6:::rbd_data.11291d0c7d7c.0000000000000003:16 v 119'548, mlcod=116'544)", 2024-09-15T00:11:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2135971589999999, 2024-09-15T00:11:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064816789999999997, 2024-09-15T00:11:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.892 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:05.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6300000000000005e-07 2024-09-15T00:11:05.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596819+0000", 2024-09-15T00:11:05.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:11:05.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596827+0000", 2024-09-15T00:11:05.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1200000000000002e-06 2024-09-15T00:11:05.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596832+0000", 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5179999999999998e-06 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596870+0000", 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.803e-05 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596891+0000", 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1746000000000002e-05 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.603253+0000", 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006361663 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.603294+0000", 2024-09-15T00:11:05.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1220000000000002e-05 2024-09-15T00:11:05.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3944 3.14 e119/118 3:2e97003a:::rbd_data.1129eb1f5358.000000000000023e:18 v 119'550, mlcod=119'546)", 2024-09-15T00:11:05.898 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:05.898 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.2054689119999997, 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048495739999999997, 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:05.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604946+0000", 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604953+0000", 2024-09-15T00:11:05.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4669999999999998e-06 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604957+0000", 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604993+0000", 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5578999999999999e-05 2024-09-15T00:11:05.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.605007+0000", 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3599e-05 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.609752+0000", 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004745768 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.609790+0000", 2024-09-15T00:11:05.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7972999999999997e-05 2024-09-15T00:11:05.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.904 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.904 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1899 3.1e e121/118 3:7d005a47:::rbd_data.112951ba645b.000000000000000d:13 v 121'473, mlcod=119'469)", 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4410386080000004, 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004557752, 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369376+0000", 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5480000000000003e-06 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369392+0000", 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5862000000000002e-05 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369400+0000", 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0290000000000005e-06 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369457+0000", 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7163999999999997e-05 2024-09-15T00:11:05.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369478+0000", 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1461999999999999e-05 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.373883+0000", 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044043550000000004 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.373929+0000", 2024-09-15T00:11:05.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6071e-05 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:25995 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3833188319999996, 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.948917703, 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 25995 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427092+0000", 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.48e-06 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427100+0000", 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.072e-06 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427113+0000", 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2845e-05 2024-09-15T00:11:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427124+0000", 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0324e-05 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427177+0000", 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3514999999999998e-05 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-15T00:11:05.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427184+0000", 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1910000000000003e-06 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.374306+0000", 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.947121671 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.374394+0000", 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7918999999999996e-05 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.374538+0000", 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014439100000000001 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.375182+0000", 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064394599999999995 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.375954+0000", 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077125800000000003 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.375985+0000", 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1483999999999999e-05 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.376009+0000", 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3606999999999999e-05 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.915 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:26031 3.1b 3:dc233cc8:::rbd_data.112951ba645b.0000000000000021:head [delete] snapc 13=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:01.411914+0000", 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3984957099999997, 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048158200000000002, 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 26031 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411914+0000", 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411914+0000", 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411916+0000", 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9130000000000001e-06 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411944+0000", 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8479e-05 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411978+0000", 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3979000000000001e-05 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411984+0000", 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:11:05.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.412041+0000", 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7176999999999998e-05 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.412223+0000", 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018224499999999999 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.412319+0000", 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6491999999999997e-05 2024-09-15T00:11:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.413928+0000", 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608827 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416677+0000", 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027486189999999999 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416706+0000", 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8719999999999999e-05 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416730+0000", 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4249000000000001e-05 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:26038 3.3 3:c4d6c711:::rbd_data.112951ba645b.000000000000007f:head [delete] snapc 13=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:01.416726+0000", 2024-09-15T00:11:05.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.393683394, 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051334279999999998, 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 26038 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416726+0000", 2024-09-15T00:11:05.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416726+0000", 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416727+0000", 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416729+0000", 2024-09-15T00:11:05.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3750000000000001e-06 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416734+0000", 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9490000000000004e-06 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416738+0000", 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416774+0000", 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6906999999999997e-05 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416882+0000", 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107736 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416959+0000", 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7106000000000003e-05 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421717+0000", 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047577059999999996 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421802+0000", 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5236999999999997e-05 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421835+0000", 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2885000000000001e-05 2024-09-15T00:11:05.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421860+0000", 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4618999999999999e-05 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:26044 3.9 3:92c2957d:::rbd_data.112951ba645b.00000000000000c8:head [delete] snapc 13=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:01.417540+0000", 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3928692590000002, 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004285779, 2024-09-15T00:11:05.927 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 26044 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.417540+0000", 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.417540+0000", 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.417541+0000", 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8100000000000002e-07 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.417545+0000", 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:11:05.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.417554+0000", 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6399999999999992e-06 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.417559+0000", 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9409999999999998e-06 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.417601+0000", 2024-09-15T00:11:05.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1424999999999997e-05 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.417711+0000", 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011014300000000001 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.417777+0000", 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6193999999999997e-05 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:05.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419880+0000", 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021024939999999999 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421737+0000", 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018576840000000001 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421787+0000", 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9716999999999998e-05 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421826+0000", 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9301999999999997e-05 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4393.0:26048 3.1c e122/118 3:388d352a:::rbd_data.112951ba645b.0000000000000131:head v 122'464, mlcod=120'462)", 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:01.419699+0000", 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.3907111289999996, 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044292389999999997, 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:05.933 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419699+0000", 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419699+0000", 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419699+0000", 2024-09-15T00:11:05.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9999999999999997e-07 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419704+0000", 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7990000000000001e-06 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419713+0000", 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0520000000000007e-06 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:05.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419718+0000", 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.69e-06 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419765+0000", 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7627999999999999e-05 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419778+0000", 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2471e-05 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.424104+0000", 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043260649999999996 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.424128+0000", 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3934000000000002e-05 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:05.937 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:05.938 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:05.939 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:05.937+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:05.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:06.039 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:06.038+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:06.046 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:11:06.049 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:06.049 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:06.049 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:11:06.049 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:06.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:06.065 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:11:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:06.084 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:11:06.084 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:06.101 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:11:06.101 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:06.118 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:11:06.118 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:06.135 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:11:06.136 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:06.139 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:11:06.139 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:11:06.140 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:06.139+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:06.167 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:06.167 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:06.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:06.260 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:06.258+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:06.324 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:06.324 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:06.333 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:06.342 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:06.341+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:06.400 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:11:06.443 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:06.441+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:06.493 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:06.493 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:06.503 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:06.547 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:06.545+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:06.612 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:06.612 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:06.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:06.645 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:06.644+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:06.746 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:06.745+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:06.768 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:06.769 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:06.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:06.842 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:11:06.840+0000 7f1c1cc1e640 -1 Falling back to public interface 2024-09-15T00:11:06.846 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:06.845+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:06.883 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:06.883 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:06.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:06.948 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:06.946+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:06.995 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:06.995 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:07.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:07.048 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:07.047+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:07.106 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:07.106 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:07.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:07.156 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:07.154+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:07.250 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:07.249+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:07.257 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:07.257 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:07.266 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:07.352 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:07.350+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:07.408 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:07.408 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:07.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:07.453 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:07.451+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:07.538 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:07.538 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:07.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:07.592 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:07.591+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:07.654 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:07.655 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:07.655 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:07.653+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:07.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:07.755 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:07.754+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:07.805 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:07.805 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:07.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:07.857 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:07.855+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:07.958 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:07.956+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:07.958 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:07.958 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:07.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:08.059 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:08.057+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:08.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:08.140 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:08.169 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:08.167+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:08.233 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:08.233 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:08.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:08.275 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:11:08.276 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:08.273+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:08.292 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:11:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:08.310 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:11:08.310 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:08.327 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:11:08.327 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:08.344 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:11:08.345 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:08.354 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:11:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:11:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:11:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322947619999994, 2024-09-15T00:11:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:11:08.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:11:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322927874000001, 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.360 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:11:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:11:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:11:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:11:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.365 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322907924000006, 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.366 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:11:08.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.371 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322880424000004, 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:11:08.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:11:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:11:08.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322863583, 2024-09-15T00:11:08.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:11:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:11:08.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:11:08.383 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322847585000005, 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:11:08.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322824596999993, 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:11:08.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322798712999997, 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:11:08.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.399 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322782286000006, 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:11:08.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322768014999994, 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:11:08.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:11:08.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.322758698000001, 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:11:08.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:11:08.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.321089685999993, 2024-09-15T00:11:08.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:11:08.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.321054622999995, 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:11:08.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:11:08.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:11:08.427 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.321029596000002, 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.428 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:11:08.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.432 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.321005287999995, 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.433 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:11:08.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.318788905000005, 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:11:08.438 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:11:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.443 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.318759533000005, 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.444 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:11:08.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:11:08.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.654256105999998, 2024-09-15T00:11:08.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:11:08.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:11:08.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5644.0:745 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:11:08.456 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.178662230999997, 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227015319999999, 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5644", 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1022470035", 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 745 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171868+0000", 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-06 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171876+0000", 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.593e-06 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171890+0000", 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4347e-05 2024-09-15T00:11:08.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171897+0000", 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8449999999999997e-06 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171949+0000", 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2206000000000001e-05 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171959+0000", 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7829999999999993e-06 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194277+0000", 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0223181750000001 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194432+0000", 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155329 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194568+0000", 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013594599999999999 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.462 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20982 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.178594541000002, 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0231241369999999, 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20982 2024-09-15T00:11:08.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:11:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171935+0000", 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1200000000000003e-07 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171938+0000", 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9550000000000001e-06 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171944+0000", 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171948+0000", 2024-09-15T00:11:08.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8519999999999997e-06 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171977+0000", 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9539999999999998e-05 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171981+0000", 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.478e-06 2024-09-15T00:11:08.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194852+0000", 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0228714160000001 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194908+0000", 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5837000000000001e-05 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.195059+0000", 2024-09-15T00:11:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015047699999999999 2024-09-15T00:11:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.469 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.469 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:08.470 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:11:08.471 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:08.469+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:08.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:08.472 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:11:08.473 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:11:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:08.473 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:11:08.571 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:08.570+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:08.626 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:08.626 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:11:08.626 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:08.635 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:08.673 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:08.672+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:08.743 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:11:08.743 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:11:08.743 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:08.752 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:08.795 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:08.793+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:08.860 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:08.860 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.832380330999996, 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.861 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:11:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.865 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.832358412999994, 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:11:08.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.832337140999996, 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:11:08.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:11:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:11:08.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:11:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:11:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:11:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:11:08.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.830595880999994, 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.878 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:11:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.830573974999993, 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.883 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.830551882999998, 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:11:08.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.892 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.830520144000005, 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:11:08.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:11:08.897 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.830221941000005, 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:11:08.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.830188235999998, 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.903 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.825886163000007, 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:11:08.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:11:08.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.911 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.825561293000007, 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.912 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:11:08.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.825150887999996, 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:11:08.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:11:08.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:11:08.921 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.825095985000004, 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:11:08.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:11:08.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.825063950000001, 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.926 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:11:08.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:11:08.930 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.823586120000002, 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:11:08.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:11:08.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.823564404999999, 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.935 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.937 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:11:08.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.939 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.823542926000002, 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:11:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:11:08.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.821529822000002, 2024-09-15T00:11:08.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:11:08.948 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.821510180999994, 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:11:08.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 98.821463632999993, 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:08.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:11:08.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:08.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:08.958 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:08.960 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:08.961 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:08.958+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:09.061 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:09.060+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:09.066 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:09.066 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:11:09.066 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:09.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:09.162 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:09.161+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:09.176 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:09.176 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:09.176 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:09.176 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:09.176 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:09.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:09.268 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:09.267+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:09.330 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.415299834000002, 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.415279527999999, 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:11:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.415257588000003, 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:11:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.413399816999998, 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:11:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.413370137999998, 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:11:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.413347045000002, 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:11:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.413321871999997, 2024-09-15T00:11:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:11:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:11:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:11:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.413301230999998, 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:11:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:11:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:11:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:11:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:11:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.413280690000001, 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:11:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.413257207999997, 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:11:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.411393552, 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:11:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:11:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.411363782999999, 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:11:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:11:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:11:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.411341604, 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.411314353999998, 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:11:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.41128776, 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:11:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.411268427000003, 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.409220017999999, 2024-09-15T00:11:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:11:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.409204785, 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:11:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.643928266000003, 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:11:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:11:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.597837595999998, 2024-09-15T00:11:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.427 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:09.428 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:11:09.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:09.430 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:09.427+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:09.529 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:09.528+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:09.577 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:11:09.577 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:09.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:09.630 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:09.629+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:09.729 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:09.729 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:09.729 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:09.729 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:09.729 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:09.730 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:09.729+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:09.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:09.831 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:09.830+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:09.882 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.997588556, 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:11:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.997549926, 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:11:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.997523989, 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.99750792099999, 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:11:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:11:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.997366826, 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:11:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.99651004, 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:11:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.99647502400001, 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:11:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:11:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:11:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.99644942099999, 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:11:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.99627515500001, 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.996044016, 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:11:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.99523207199999, 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:11:09.930 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:11:09.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:11:09.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.995152332, 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:09.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.939 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.994947307, 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:11:09.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:11:09.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.99473577400001, 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.944 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:11:09.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.949 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.949 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:09.949 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.992179767, 2024-09-15T00:11:09.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:11:09.949 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.949 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:09.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:11:09.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.990196661, 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.99017709, 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:11:09.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:09.963 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.719434239999998, 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:11:09.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.813682745000001, 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.968 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:11:09.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.972 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.196595305000002, 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:09.973 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:09.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:11:09.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:11:09.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:09.978 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:09.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:09.980 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:09.978+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:10.080 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:10.079+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:10.123 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:10.123 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:10.123 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:11:10.123 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:10.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:10.181 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:10.180+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:10.274 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:10.274 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:10.274 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:10.274 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:10.275 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:10.282 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:10.280+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:10.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:10.383 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:10.382+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:10.433 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1893 3.1e e119/118 3:7c90e45f:::rbd_data.11291f9c3154.000000000000003a:1d v 119'463, mlcod=119'459)", 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002171828, 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005948208, 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.37e-07 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428523+0000", 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2139999999999996e-06 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428532+0000", 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7229999999999999e-06 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428539+0000", 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3289999999999999e-06 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428578+0000", 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8792999999999999e-05 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428595+0000", 2024-09-15T00:11:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7669e-05 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.434405+0000", 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058096780000000004 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.434462+0000", 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7065000000000003e-05 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3933 3.15 e119/118 3:ad39a9e7:::rbd_data.11291d0c7d7c.0000000000000000:16 v 119'404, mlcod=115'401)", 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.000847509, 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075329059999999998, 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429839+0000", 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5799999999999999e-07 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429844+0000", 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7649999999999999e-06 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429855+0000", 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0644e-05 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429862+0000", 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.477e-06 2024-09-15T00:11:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429903+0000", 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1693999999999998e-05 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429931+0000", 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7699e-05 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.437329+0000", 2024-09-15T00:11:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073982270000000003 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.437371+0000", 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1742000000000001e-05 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1895 3.1e e119/118 3:7d7cb6ca:::rbd_data.11294a896ed0.0000000000000010:22 v 119'467, mlcod=119'463)", 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.990349473, 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007846713, 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440341+0000", 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7350000000000004e-06 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440349+0000", 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6299999999999998e-06 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440357+0000", 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1710000000000002e-06 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440413+0000", 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5980000000000003e-05 2024-09-15T00:11:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440432+0000", 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8847000000000001e-05 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.448132+0000", 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007700386 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.448183+0000", 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0559000000000002e-05 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3934 3.15 e119/118 3:a9ed06fd:::rbd_data.112986cdeff1.0000000000000014:1b v 119'406, mlcod=116'402)", 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.989866579999999, 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010330291, 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440820+0000", 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3099999999999997e-07 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440825+0000", 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.186e-06 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440836+0000", 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.092e-05 2024-09-15T00:11:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440842+0000", 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3890000000000002e-06 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440903+0000", 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1393e-05 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440921+0000", 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7975e-05 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451109+0000", 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010187391000000001 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451149+0000", 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0405999999999999e-05 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1897 3.1e e119/118 3:7a2239c1:::rbd_data.1129466cbf24.000000000000000a:27 v 119'471, mlcod=119'465)", 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.979702349, 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005507311, 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450984+0000", 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450987+0000", 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9409999999999999e-06 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450992+0000", 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2789999999999998e-06 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450995+0000", 2024-09-15T00:11:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0740000000000001e-06 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451734+0000", 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073883599999999998 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451756+0000", 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2158999999999999e-05 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.456450+0000", 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004693666 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.456491+0000", 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0976000000000002e-05 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3936 3.15 e119/118 3:a9d1a5ec:::rbd_data.1129696a4676.0000000000000006:24 v 119'410, mlcod=119'406)", 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.969055185, 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006441413, 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461631+0000", 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4899999999999995e-07 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461636+0000", 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6659999999999999e-06 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461644+0000", 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5269999999999998e-06 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461653+0000", 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5620000000000004e-06 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461705+0000", 2024-09-15T00:11:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2045000000000001e-05 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461724+0000", 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.893e-05 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.468031+0000", 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006307773 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.468072+0000", 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0361000000000002e-05 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3937 3.15 e119/118 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:27 v 119'412, mlcod=119'408)", 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.956484995, 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077903419999999996, 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474207+0000", 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0820000000000004e-06 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474218+0000", 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.111e-05 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474225+0000", 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7830000000000001e-06 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474267+0000", 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1644999999999999e-05 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474288+0000", 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0693999999999999e-05 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.481944+0000", 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076562280000000002 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.481991+0000", 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7083000000000003e-05 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5281 3.e e119/118 3:771a58a1:::rbd_data.1129eb1f5358.0000000000000007:18 v 119'285, mlcod=110'280)", 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.944167922, 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064605180000000002, 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:11:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486523+0000", 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486531+0000", 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2570000000000001e-06 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486535+0000", 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4170000000000002e-06 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486578+0000", 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099000000000002e-05 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486599+0000", 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0735999999999999e-05 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.492941+0000", 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063422289999999996 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.492978+0000", 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7045000000000002e-05 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5282 3.e e119/118 3:701c32a5:::rbd_data.112929d537d9.00000000000001c7:1e v 119'287, mlcod=119'283)", 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.934123962999999, 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046705490000000004, 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496562+0000", 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2099999999999996e-07 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496568+0000", 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0170000000000002e-06 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496577+0000", 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8939999999999999e-06 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496586+0000", 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5080000000000005e-06 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496625+0000", 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9140999999999997e-05 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.496645+0000", 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0248999999999999e-05 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.501202+0000", 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045572820000000002 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.501232+0000", 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9836999999999999e-05 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5283 3.e e119/118 3:7642e0f1:::rbd_data.112929d537d9.000000000000002c:1e v 119'289, mlcod=119'283)", 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.933465342, 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064221740000000001, 2024-09-15T00:11:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497221+0000", 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7299999999999995e-07 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497226+0000", 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0239999999999997e-06 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497238+0000", 2024-09-15T00:11:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1746000000000001e-05 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497244+0000", 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6640000000000001e-06 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497287+0000", 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3130999999999998e-05 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497307+0000", 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9740000000000001e-05 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.503604+0000", 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062964780000000003 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.503642+0000", 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8717999999999998e-05 2024-09-15T00:11:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3943 3.14 e119/118 3:2a8444a6:::rbd_data.11291d0c7d7c.0000000000000003:16 v 119'548, mlcod=116'544)", 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.833872997, 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064816789999999997, 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6300000000000005e-07 2024-09-15T00:11:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596819+0000", 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596827+0000", 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1200000000000002e-06 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596832+0000", 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5179999999999998e-06 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596870+0000", 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.803e-05 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596891+0000", 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1746000000000002e-05 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.603253+0000", 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006361663 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.603294+0000", 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1220000000000002e-05 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3944 3.14 e119/118 3:2e97003a:::rbd_data.1129eb1f5358.000000000000023e:18 v 119'550, mlcod=119'546)", 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.82574475, 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048495739999999997, 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604941+0000", 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604946+0000", 2024-09-15T00:11:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604953+0000", 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4669999999999998e-06 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604957+0000", 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.604993+0000", 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5578999999999999e-05 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.605007+0000", 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3599e-05 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.609752+0000", 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004745768 2024-09-15T00:11:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.609790+0000", 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7972999999999997e-05 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1899 3.1e e121/118 3:7d005a47:::rbd_data.112951ba645b.000000000000000d:13 v 121'473, mlcod=119'469)", 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.061314446000001, 2024-09-15T00:11:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004557752, 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369371+0000", 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369376+0000", 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5480000000000003e-06 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369392+0000", 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5862000000000002e-05 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369400+0000", 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0290000000000005e-06 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369457+0000", 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7163999999999997e-05 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.369478+0000", 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1461999999999999e-05 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.373883+0000", 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044043550000000004 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.373929+0000", 2024-09-15T00:11:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6071e-05 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:25995 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.00359467, 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.948917703, 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 25995 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427092+0000", 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.48e-06 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427100+0000", 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.072e-06 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427113+0000", 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2845e-05 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427124+0000", 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0324e-05 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427177+0000", 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3514999999999998e-05 2024-09-15T00:11:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427184+0000", 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1910000000000003e-06 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.374306+0000", 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.947121671 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.374394+0000", 2024-09-15T00:11:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7918999999999996e-05 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.374538+0000", 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014439100000000001 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.375182+0000", 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064394599999999995 2024-09-15T00:11:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.375954+0000", 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077125800000000003 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.375985+0000", 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1483999999999999e-05 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.376009+0000", 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3606999999999999e-05 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:26031 3.1b 3:dc233cc8:::rbd_data.112951ba645b.0000000000000021:head [delete] snapc 13=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:01.411914+0000", 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0187715480000001, 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048158200000000002, 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 26031 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411914+0000", 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411914+0000", 2024-09-15T00:11:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411916+0000", 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9130000000000001e-06 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411944+0000", 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8479e-05 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411978+0000", 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3979000000000001e-05 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.411984+0000", 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1200000000000001e-06 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.412041+0000", 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7176999999999998e-05 2024-09-15T00:11:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.412223+0000", 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018224499999999999 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.412319+0000", 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6491999999999997e-05 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.413928+0000", 2024-09-15T00:11:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608827 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416677+0000", 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027486189999999999 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416706+0000", 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8719999999999999e-05 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416730+0000", 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4249000000000001e-05 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:26038 3.3 3:c4d6c711:::rbd_data.112951ba645b.000000000000007f:head [delete] snapc 13=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:01.416726+0000", 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0139592319999995, 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051334279999999998, 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 26038 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416726+0000", 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416726+0000", 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416727+0000", 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416729+0000", 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3750000000000001e-06 2024-09-15T00:11:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416734+0000", 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9490000000000004e-06 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416738+0000", 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5279999999999999e-06 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416774+0000", 2024-09-15T00:11:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6906999999999997e-05 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416882+0000", 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107736 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.416959+0000", 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7106000000000003e-05 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421717+0000", 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047577059999999996 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421802+0000", 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5236999999999997e-05 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421835+0000", 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2885000000000001e-05 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.421860+0000", 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4618999999999999e-05 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4393.0:26048 3.1c e122/118 3:388d352a:::rbd_data.112951ba645b.0000000000000131:head v 122'464, mlcod=120'462)", 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:01.419699+0000", 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0109869669999991, 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044292389999999997, 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419699+0000", 2024-09-15T00:11:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419699+0000", 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419699+0000", 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9999999999999997e-07 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419704+0000", 2024-09-15T00:11:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7990000000000001e-06 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419713+0000", 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0520000000000007e-06 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419718+0000", 2024-09-15T00:11:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.69e-06 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419765+0000", 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7627999999999999e-05 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.419778+0000", 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2471e-05 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.424104+0000", 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043260649999999996 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.424128+0000", 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3934000000000002e-05 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5491 3.e e129/118 3:7756d743:::rbd_data.11296e862a2c.0000000000000057:c v 129'304, mlcod=129'300)", 2024-09-15T00:11:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:08.449218+0000", 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.9814678570000002, 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047818449999999998, 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.449218+0000", 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.449218+0000", 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.449218+0000", 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9299999999999997e-07 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.449224+0000", 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8599999999999998e-06 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.449235+0000", 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0456e-05 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.449241+0000", 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2929999999999997e-06 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.449297+0000", 2024-09-15T00:11:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6159000000000002e-05 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.449318+0000", 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0520999999999999e-05 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.453945+0000", 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046269609999999997 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:08.454000+0000", 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4901999999999997e-05 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:27417 3.1a 3:5f9784b4:::rbd_data.112910301a74.0000000000000003:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:09.500472+0000", 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.93021327399999998, 2024-09-15T00:11:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052738500000000001, 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 27417 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500472+0000", 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500472+0000", 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500474+0000", 2024-09-15T00:11:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3850000000000001e-06 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500480+0000", 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2600000000000002e-06 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500495+0000", 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4908e-05 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500504+0000", 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1870000000000003e-06 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500565+0000", 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0792999999999999e-05 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500858+0000", 2024-09-15T00:11:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029332000000000002 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500961+0000", 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103322 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.502687+0000", 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001725589 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505688+0000", 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030005980000000002 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505719+0000", 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1627000000000001e-05 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505746+0000", 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6860999999999999e-05 2024-09-15T00:11:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:27419 3.9 3:926cd00d:::rbd_data.112910301a74.0000000000000005:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:09.500527+0000", 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.93015816900000003, 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052930809999999998, 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 27419 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500527+0000", 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500527+0000", 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500528+0000", 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0100000000000005e-07 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500531+0000", 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7870000000000002e-06 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500539+0000", 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8960000000000003e-06 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500545+0000", 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9909999999999999e-06 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500644+0000", 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9451999999999996e-05 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500942+0000", 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029790399999999999 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.501476+0000", 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053428700000000004 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.502813+0000", 2024-09-15T00:11:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001337038 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505729+0000", 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029160050000000002 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505782+0000", 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2472000000000002e-05 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505820+0000", 2024-09-15T00:11:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8748000000000002e-05 2024-09-15T00:11:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:10.537 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:10.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:10.539 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:10.538+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:10.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:10.640 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:10.639+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:10.648 INFO:teuthology.orchestra.run.smithi027.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T00:11:10.652 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:10.652 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:11:10.652 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:10.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:10.666 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:11:10.666 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:10.683 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:11:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:10.700 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:11:10.701 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:10.717 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:11:10.717 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:10.734 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:11:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:10.737 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T00:11:10.737 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T00:11:10.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:10.741 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:10.738+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:10.774 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:10.774 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:10.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:10.841 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:10.840+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:no valid command found; 10 closest matches: 2024-09-15T00:11:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:0 2024-09-15T00:11:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:1 2024-09-15T00:11:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2 2024-09-15T00:11:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:abort 2024-09-15T00:11:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:assert 2024-09-15T00:11:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:bluefs debug_inject_read_zeros 2024-09-15T00:11:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:bluefs files list 2024-09-15T00:11:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:bluefs stats 2024-09-15T00:11:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:bluestore allocator dump block 2024-09-15T00:11:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:bluestore allocator fragmentation block 2024-09-15T00:11:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:admin_socket: invalid command 2024-09-15T00:11:10.884 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T00:11:10.884 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T00:11:10.917 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:11:10.915+0000 7f1c1cc1e640 -1 osd.0 55 log_to_monitors true 2024-09-15T00:11:10.935 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:10.935 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:10.942 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:10.941+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:10.944 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:11.043 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:11.041+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:11.091 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:11.091 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:11.100 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:11.144 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:11.143+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:11.207 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:11.207 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:11.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:11.244 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:11.242+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:11.346 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:11.344+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:11.363 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:11.363 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:11.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:11.438 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T00:11:11.446 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:11.444+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:11.486 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:11:11.484+0000 7f1c109cd640 -1 osd.0 132 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T00:11:11.496 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:11.496 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:11.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:11.547 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:11.546+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:11.647 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:11.645+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:11.648 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:11.648 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:11.656 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:11.748 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:11.746+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:11.800 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:11.800 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T00:11:11.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:11.862 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:11.861+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:11.950 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:11.949+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:11.959 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:11.959 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:11.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:12.051 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:12.048+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:12.071 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:12.071 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:12.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:12.163 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:12.161+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:12.224 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:12.224 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:12.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:12.273 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:12.270+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:12.353 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:12.350+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:12.374 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:12.374 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:12.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:12.468 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:12.465+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:12.525 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:12.526 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:12.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:12.575 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:12.572+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:12.655 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:12.653+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:12.676 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:12.677 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:12.685 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:12.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T00:11:12.756 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:12.755+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:12.810 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:12.810 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:12.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:12.865 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:12.864+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:12.878 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:11:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-15T00:11:12.898 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:11:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:12.917 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:11:12.917 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:12.936 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:11:12.936 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:12.939 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:12.939 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:12.939 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:11:12.939 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:11:12.939 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:12.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:12.953 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:11:12.953 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:12.958 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:12.957+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:12.972 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:11:12.972 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:13.060 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:13.058+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:13.065 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:13.065 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:11:13.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:11:13.065 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:11:13.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.065 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2658 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 2981888~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.065 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:13.065 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.035016, 2024-09-15T00:11:13.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060198203999999998, 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2658 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:13.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027582+0000", 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9700000000000002e-07 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027584+0000", 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.962e-06 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027590+0000", 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0510000000000002e-06 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027595+0000", 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1830000000000002e-06 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087563+0000", 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059968064000000001 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087669+0000", 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105967 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087779+0000", 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011058 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2659 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3031040~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.03499625400001, 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060488055999999998, 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.070 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2659 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.071 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027601+0000", 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6899999999999998e-07 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027603+0000", 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.03e-06 2024-09-15T00:11:13.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027610+0000", 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7320000000000001e-06 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027615+0000", 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4619999999999998e-06 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087816+0000", 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060201428000000001 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087949+0000", 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013269900000000001 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088089+0000", 2024-09-15T00:11:13.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014033599999999999 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2660 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3080192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.034976304, 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060705595000000001, 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2660 2024-09-15T00:11:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:11:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027621+0000", 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0600000000000001e-07 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027624+0000", 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2309999999999999e-06 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027630+0000", 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.336e-06 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027642+0000", 2024-09-15T00:11:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2247e-05 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088121+0000", 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060478849000000001 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088215+0000", 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3776000000000005e-05 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088327+0000", 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011175000000000001 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2661 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3129344~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.034948804, 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060936465000000002, 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.080 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2661 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027648+0000", 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027649+0000", 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027651+0000", 2024-09-15T00:11:13.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.978e-06 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027656+0000", 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5709999999999997e-06 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027661+0000", 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2579999999999997e-06 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088363+0000", 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060702627000000002 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088459+0000", 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5674000000000006e-05 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088585+0000", 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000125974 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.084 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2662 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3178496~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.03493196300001, 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061132552, 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.085 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2662 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027665+0000", 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027666+0000", 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1e-07 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027667+0000", 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109999999999999e-06 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027672+0000", 2024-09-15T00:11:13.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2170000000000002e-06 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027677+0000", 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0620000000000002e-06 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088639+0000", 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060962711000000003 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088733+0000", 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3992000000000004e-05 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088798+0000", 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4648999999999997e-05 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.089 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2663 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3227648~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.034915965, 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061300458000000002, 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2663 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027681+0000", 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027682+0000", 2024-09-15T00:11:13.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.27e-07 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027688+0000", 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4400000000000002e-06 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027694+0000", 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.699e-06 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027698+0000", 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9940000000000002e-06 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088829+0000", 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061130991000000003 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088887+0000", 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8349e-05 2024-09-15T00:11:13.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.088982+0000", 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4658000000000001e-05 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2664 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3276800~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.034892977, 2024-09-15T00:11:13.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061513756000000003, 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2664 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:11:13.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027704+0000", 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027705+0000", 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.89e-07 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.096 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027712+0000", 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6909999999999997e-06 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027719+0000", 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6730000000000003e-06 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027724+0000", 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6870000000000002e-06 2024-09-15T00:11:13.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089013+0000", 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061288830000000002 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089120+0000", 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010721300000000001 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089218+0000", 2024-09-15T00:11:13.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8272999999999999e-05 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2665 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3325952~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.034867093, 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061734244000000001, 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2665 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027730+0000", 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027731+0000", 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9499999999999998e-07 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027732+0000", 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027738+0000", 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3249999999999998e-06 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027742+0000", 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0910000000000003e-06 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089253+0000", 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061511130999999997 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089349+0000", 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5910999999999997e-05 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089464+0000", 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115637 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2666 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3375104~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.103 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.034850666, 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061952563000000002, 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2666 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027747+0000", 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1199999999999999e-07 2024-09-15T00:11:13.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027749+0000", 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.663e-06 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027754+0000", 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1420000000000001e-06 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027758+0000", 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1540000000000004e-06 2024-09-15T00:11:13.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089497+0000", 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061739364999999997 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089583+0000", 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5684000000000002e-05 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089699+0000", 2024-09-15T00:11:13.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116243 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2667 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.034836395, 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062182732999999997, 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.108 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2667 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027761+0000", 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0699999999999999e-07 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027762+0000", 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.127e-06 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027765+0000", 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8650000000000001e-06 2024-09-15T00:11:13.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027767+0000", 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3070000000000001e-06 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089743+0000", 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061975352999999997 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089832+0000", 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9064e-05 2024-09-15T00:11:13.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089944+0000", 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011181 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2668 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3473408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.03482707800001, 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.062407029000000003, 2024-09-15T00:11:13.112 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2668 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027770+0000", 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029400+0000", 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001629674 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6600000000000003e-06 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029417+0000", 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2913e-05 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029425+0000", 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9650000000000008e-06 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.089980+0000", 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060554152 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090069+0000", 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9079999999999994e-05 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090177+0000", 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000108585 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.116 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2669 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3522560~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.033158066, 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060975910000000001, 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2669 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029439+0000", 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029440+0000", 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4499999999999997e-07 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029443+0000", 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.931e-06 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029451+0000", 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9289999999999998e-06 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029466+0000", 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5153999999999999e-05 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090213+0000", 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060747096 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090297+0000", 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4134000000000005e-05 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090415+0000", 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118121 2024-09-15T00:11:13.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2670 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3571712~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.033123003, 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061174251999999998, 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2670 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029474+0000", 2024-09-15T00:11:13.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029475+0000", 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8699999999999995e-07 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029478+0000", 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.926e-06 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029486+0000", 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1589999999999993e-06 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029491+0000", 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.604e-06 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090448+0000", 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060956042000000002 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090538+0000", 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0919000000000001e-05 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090649+0000", 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110115 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.125 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2671 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3620864~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.03309797599999, 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061369604000000001, 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2671 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029499+0000", 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029500+0000", 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029503+0000", 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1659999999999998e-06 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029510+0000", 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8140000000000003e-06 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029515+0000", 2024-09-15T00:11:13.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5180000000000004e-06 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090682+0000", 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061167100000000002 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090767+0000", 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5025000000000007e-05 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090869+0000", 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101543 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2672 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3670016~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.033073668, 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.061572066000000002, 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2672 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029524+0000", 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031583+0000", 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020597990000000002 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031588+0000", 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0699999999999997e-06 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031600+0000", 2024-09-15T00:11:13.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1895999999999999e-05 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031608+0000", 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3039999999999999e-06 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090903+0000", 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059295704999999997 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.090985+0000", 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1318999999999998e-05 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091096+0000", 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000110973 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2676 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3866624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:11:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.030857285, 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060133929000000003, 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2676 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031740+0000", 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8100000000000003e-07 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031750+0000", 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1980000000000007e-06 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031757+0000", 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.968e-06 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031762+0000", 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1410000000000004e-06 2024-09-15T00:11:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091766+0000", 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.060004061999999997 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091818+0000", 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1847e-05 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091874+0000", 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6232e-05 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:2677 3.4 3:22886c08:::rbd_data.1129915ddc5d.0000000000000000:f [read 3915776~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.030827913, 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.06035724, 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.139 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 2677 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.031769+0000", 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033400+0000", 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001630367 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033405+0000", 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1989999999999997e-06 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033417+0000", 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2362e-05 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033425+0000", 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6320000000000008e-06 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091963+0000", 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058538222000000001 2024-09-15T00:11:13.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092020+0000", 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6805000000000002e-05 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092127+0000", 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106653 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5548.0:675 3.4 3:20e60685:::rbd_data.1129466cbf24.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 54.366324486000003, 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.069158054999999996, 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5548", 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T00:11:13.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696273+0000", 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.696274+0000", 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7499999999999999e-07 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740884+0000", 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.044610360000000002 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740912+0000", 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7858999999999999e-05 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.740922+0000", 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.876e-06 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741020+0000", 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8023000000000007e-05 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.741142+0000", 2024-09-15T00:11:13.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121879 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.742707+0000", 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015652730000000001 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765028+0000", 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.022321032000000001 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op_commit", 2024-09-15T00:11:13.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765329+0000", 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00030117000000000002 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765377+0000", 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7624999999999999e-05 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:18.765431+0000", 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4082999999999997e-05 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.149 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.5644.0:745 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.890730611000002, 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227015319999999, 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.5644", 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1022470035", 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 745 2024-09-15T00:11:13.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171867+0000", 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171868+0000", 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-06 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171876+0000", 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.593e-06 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171890+0000", 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4347e-05 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171897+0000", 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8449999999999997e-06 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171949+0000", 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2206000000000001e-05 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171959+0000", 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7829999999999993e-06 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194277+0000", 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0223181750000001 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194432+0000", 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155329 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194568+0000", 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013594599999999999 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.154 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4393.0:20982 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e90)", 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.890662921000001, 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0231241369999999, 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4393", 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 20982 2024-09-15T00:11:13.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171934+0000", 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171935+0000", 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1200000000000003e-07 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171938+0000", 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9550000000000001e-06 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171944+0000", 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-06 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171948+0000", 2024-09-15T00:11:13.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8519999999999997e-06 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171977+0000", 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9539999999999998e-05 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:28.171981+0000", 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.478e-06 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194852+0000", 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0228714160000001 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.194908+0000", 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5837000000000001e-05 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:10:29.195059+0000", 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015047699999999999 2024-09-15T00:11:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.162 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.162 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.162 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:13.164 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:13.164 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:13.162+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:13.263 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:13.262+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:13.311 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:13.312 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:13.312 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T00:11:13.312 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:13.320 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:13.366 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:13.365+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:13.465 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:13.463+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:13.470 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:13.470 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T00:11:13.470 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T00:11:13.470 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T00:11:13.470 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:13.479 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:13.568 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:13.565+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:13.588 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:13.588 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T00:11:13.588 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T00:11:13.588 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T00:11:13.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.588 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3502 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6209536~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.560338227, 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058949669000000003, 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3502 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.589 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.590 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025721+0000", 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9999999999999998e-07 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025723+0000", 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1229999999999998e-06 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025730+0000", 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5769999999999999e-06 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025735+0000", 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7029999999999997e-06 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084502+0000", 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058767076000000001 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084584+0000", 2024-09-15T00:11:13.592 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2187000000000004e-05 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084671+0000", 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6602999999999997e-05 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3504 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6258688~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.560316309, 2024-09-15T00:11:13.593 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059124746999999998, 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3504 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.594 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025743+0000", 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9400000000000001e-07 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025745+0000", 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0260000000000002e-06 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025752+0000", 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7270000000000001e-06 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025757+0000", 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2059999999999998e-06 2024-09-15T00:11:13.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084702+0000", 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058944446999999997 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084782+0000", 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0486000000000002e-05 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.597 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084868+0000", 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5561000000000003e-05 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3506 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6324224~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.560295037, 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059315148999999998, 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3506 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025764+0000", 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025765+0000", 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2899999999999999e-07 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025767+0000", 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0609999999999998e-06 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025773+0000", 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7689999999999999e-06 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.025786+0000", 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2333e-05 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084897+0000", 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059111652000000001 2024-09-15T00:11:13.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.084977+0000", 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0087999999999996e-05 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.085079+0000", 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000101817 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3511 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6569984~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.558553777, 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059000980000000001, 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3511 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.603 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027505+0000", 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027506+0000", 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3000000000000001e-07 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027508+0000", 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5689999999999999e-06 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027515+0000", 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5919999999999997e-06 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027520+0000", 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2399999999999998e-06 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086332+0000", 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058811474000000002 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086418+0000", 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6601000000000007e-05 2024-09-15T00:11:13.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086506+0000", 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8073999999999997e-05 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3512 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6619136~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.558531871, 2024-09-15T00:11:13.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059176758000000003, 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3512 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:11:13.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027527+0000", 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027528+0000", 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5900000000000003e-07 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027530+0000", 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2709999999999999e-06 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027537+0000", 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4089999999999997e-06 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027543+0000", 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086538+0000", 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058995028999999997 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086619+0000", 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1883000000000002e-05 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086704+0000", 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4667999999999996e-05 2024-09-15T00:11:13.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3513 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6668288~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.558509779, 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059351912, 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3513 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027549+0000", 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027550+0000", 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2e-07 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027559+0000", 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027567+0000", 2024-09-15T00:11:13.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7300000000000005e-06 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027573+0000", 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.224e-06 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086736+0000", 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059162537000000001 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086814+0000", 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8029999999999997e-05 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086901+0000", 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7563000000000002e-05 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3514 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6717440~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:13.616 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.55847804, 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059514193, 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3514 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027581+0000", 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027836+0000", 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025512400000000002 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027840+0000", 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0609999999999997e-06 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027851+0000", 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.065e-05 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027867+0000", 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6495999999999999e-05 2024-09-15T00:11:13.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.086931+0000", 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059063413000000002 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087007+0000", 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6155999999999994e-05 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087095+0000", 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8293000000000003e-05 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3515 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6766592~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.558179837, 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059409864, 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3515 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027879+0000", 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027880+0000", 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2900000000000004e-07 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027883+0000", 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.993e-06 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027891+0000", 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3659999999999992e-06 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027904+0000", 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2302e-05 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087126+0000", 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059222417999999999 2024-09-15T00:11:13.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087202+0000", 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6357000000000001e-05 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087289+0000", 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6898999999999996e-05 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3516 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 6815744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.558146132, 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059587200999999999, 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3516 2024-09-15T00:11:13.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.027913+0000", 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029399+0000", 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00148584 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029405+0000", 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6969999999999998e-06 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029421+0000", 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6487e-05 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.029429+0000", 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4429999999999997e-06 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087318+0000", 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057889399000000001 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087414+0000", 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6520000000000004e-05 2024-09-15T00:11:13.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.087500+0000", 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5815000000000004e-05 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3529 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7454720~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.553844059, 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059051310000000003, 2024-09-15T00:11:13.630 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3529 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032215+0000", 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032216+0000", 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032218+0000", 2024-09-15T00:11:13.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4210000000000002e-06 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032225+0000", 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8730000000000001e-06 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032230+0000", 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1320000000000002e-06 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091049+0000", 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058818991000000001 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091129+0000", 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0525000000000006e-05 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091266+0000", 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013695499999999999 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.634 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3530 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7503872~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.553519189, 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058990885, 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.635 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3530 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032540+0000", 2024-09-15T00:11:13.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032541+0000", 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1800000000000005e-07 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032879+0000", 2024-09-15T00:11:13.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033815400000000002 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032893+0000", 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4144e-05 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032908+0000", 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5223e-05 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091305+0000", 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058396275999999997 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091383+0000", 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8570000000000002e-05 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091531+0000", 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014779999999999999 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.640 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3532 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7602176~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.553108784, 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059126017000000003, 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3532 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032950+0000", 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032951+0000", 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032953+0000", 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4099999999999998e-06 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032976+0000", 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2858999999999999e-05 2024-09-15T00:11:13.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.032994+0000", 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7883000000000001e-05 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091831+0000", 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058837458000000002 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.091929+0000", 2024-09-15T00:11:13.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7204e-05 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092076+0000", 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014782 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3533 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7651328~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:11:13.645 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.553053881, 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059334630999999999, 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3533 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:11:13.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033005+0000", 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033006+0000", 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033009+0000", 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2899999999999998e-06 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033017+0000", 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.7009999999999996e-06 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033029+0000", 2024-09-15T00:11:13.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.168e-05 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092111+0000", 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059082098 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092202+0000", 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1107999999999996e-05 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092340+0000", 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138174 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3534 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7700480~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.55302184600001, 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059574805000000002, 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.650 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3534 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.033037+0000", 2024-09-15T00:11:13.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034398+0000", 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001360361 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034417+0000", 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.927e-05 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034431+0000", 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4346e-05 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034439+0000", 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5079999999999999e-06 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092372+0000", 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057933064999999999 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092462+0000", 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0136999999999994e-05 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.092612+0000", 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000150118 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.654 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3537 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7847936~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.55154401599999, 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058961659999999999, 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3537 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034515+0000", 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034516+0000", 2024-09-15T00:11:13.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034518+0000", 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3640000000000002e-06 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034525+0000", 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6170000000000003e-06 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034530+0000", 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3839999999999996e-06 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093202+0000", 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058671829000000002 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093311+0000", 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010911700000000001 2024-09-15T00:11:13.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093477+0000", 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016594400000000001 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3538 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7897088~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:13.659 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.55152230100001, 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059726540000000002, 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3538 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034537+0000", 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0900000000000002e-07 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034539+0000", 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0729999999999999e-06 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034546+0000", 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6959999999999999e-06 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034551+0000", 2024-09-15T00:11:13.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093512+0000", 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058960239999999997 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093597+0000", 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5359999999999996e-05 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094263+0000", 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066654599999999996 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3539 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 7946240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.55150082199999, 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059445802999999998, 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3539 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.034558+0000", 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036416+0000", 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001857477 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036421+0000", 2024-09-15T00:11:13.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3190000000000002e-06 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036431+0000", 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4080000000000004e-06 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036437+0000", 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8190000000000003e-06 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093781+0000", 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.057343133999999997 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.093864+0000", 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3961999999999993e-05 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.094004+0000", 2024-09-15T00:11:13.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139684 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3545 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8241152~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.54948771799999, 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059019879999999997, 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.669 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3545 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036571+0000", 2024-09-15T00:11:13.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036572+0000", 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.77e-07 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036574+0000", 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.199e-06 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036580+0000", 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9259999999999997e-06 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036584+0000", 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5199999999999999e-06 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095345+0000", 2024-09-15T00:11:13.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058760158999999999 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095439+0000", 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4042999999999994e-05 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095591+0000", 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015265600000000001 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.673 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3546 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8290304~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.549468077, 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059274014, 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.674 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3546 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036591+0000", 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036592+0000", 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1100000000000001e-07 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036607+0000", 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5517000000000001e-05 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036621+0000", 2024-09-15T00:11:13.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4416999999999999e-05 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036628+0000", 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1600000000000003e-06 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095628+0000", 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.058999971999999998 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095721+0000", 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3448000000000004e-05 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.095865+0000", 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144089 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.678 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4899.0:3547 3.18 3:1db14ffa:::rbd_data.1129915ddc5d.0000000000000003:f [read 8339456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 103.549421529, 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059759898999999998, 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4899", 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.92:0/1779602099", 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 3547 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T00:11:13.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036638+0000", 2024-09-15T00:11:13.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.89e-07 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036641+0000", 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9170000000000002e-06 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036652+0000", 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1122e-05 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T00:11:13.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.036658+0000", 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096160+0000", 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059501989999999998 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096258+0000", 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7942000000000005e-05 2024-09-15T00:11:13.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T00:09:30.096398+0000", 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139636 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T00:11:13.683 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:13.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:13.685 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:13.683+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:13.785 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:13.782+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:13.799 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:11:13.799 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:13.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:13.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T00:11:13.886 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:13.885+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:13.958 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:13.959 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:13.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:14.011 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:14.010+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:14.073 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5956 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3420160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.158322970999997, 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042485215, 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5956 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912149+0000", 2024-09-15T00:11:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912152+0000", 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.542e-06 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912158+0000", 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6749999999999999e-06 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912162+0000", 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4649999999999996e-06 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954425+0000", 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042262632000000001 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954519+0000", 2024-09-15T00:11:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4611000000000006e-05 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954634+0000", 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114943 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5957 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3461120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.158302665000001, 2024-09-15T00:11:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811900999999999, 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5957 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912169+0000", 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912170+0000", 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912172+0000", 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912177+0000", 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5860000000000004e-06 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912182+0000", 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6240000000000001e-06 2024-09-15T00:11:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954696+0000", 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042513721999999997 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954820+0000", 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124635 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.954981+0000", 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016089899999999999 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5958 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3502080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.158280724999997, 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042995674999999997, 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5958 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.912191+0000", 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913946+0000", 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001754565 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913950+0000", 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6090000000000001e-06 2024-09-15T00:11:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913955+0000", 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.913959+0000", 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0249999999999996e-06 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955019+0000", 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041059633999999998 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955099+0000", 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0123000000000005e-05 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.955187+0000", 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8183999999999995e-05 2024-09-15T00:11:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5966 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3829760~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.156422954, 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042469054999999999, 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5966 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914049+0000", 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914050+0000", 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914051+0000", 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.632e-06 2024-09-15T00:11:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914056+0000", 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914061+0000", 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956351+0000", 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042289951999999999 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956428+0000", 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7276999999999999e-05 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956518+0000", 2024-09-15T00:11:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9926999999999995e-05 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5967 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3870720~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.156393274999999, 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042542598000000001, 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5967 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914079+0000", 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914081+0000", 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-15T00:11:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914087+0000", 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8030000000000002e-06 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914094+0000", 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2670000000000003e-06 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956532+0000", 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042438259999999998 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956573+0000", 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0787000000000001e-05 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956621+0000", 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8801999999999998e-05 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5968 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3911680~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.156370182000003, 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042723616999999998, 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5968 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914102+0000", 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914104+0000", 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914111+0000", 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914118+0000", 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6420000000000001e-06 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956653+0000", 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042535256 2024-09-15T00:11:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956730+0000", 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7398000000000007e-05 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956826+0000", 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5225999999999999e-05 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5969 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3952640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.156345008999999, 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042883325999999999, 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5969 2024-09-15T00:11:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914127+0000", 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914129+0000", 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914136+0000", 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0729999999999999e-06 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914141+0000", 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0509999999999996e-06 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956875+0000", 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042733487000000001 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.956934+0000", 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9551e-05 2024-09-15T00:11:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957010+0000", 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5993000000000002e-05 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5970 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 3993600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.156324368, 2024-09-15T00:11:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043010650999999997, 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5970 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914148+0000", 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914150+0000", 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914155+0000", 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5799999999999999e-06 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914160+0000", 2024-09-15T00:11:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1390000000000003e-06 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957037+0000", 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042876758000000001 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957101+0000", 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4249000000000001e-05 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957158+0000", 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6873999999999998e-05 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5971 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4034560~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.156303827000002, 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043125863, 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5971 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914168+0000", 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914169+0000", 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914170+0000", 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7540000000000001e-06 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914180+0000", 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9550000000000004e-06 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914185+0000", 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9289999999999997e-06 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957187+0000", 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043001969000000001 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957239+0000", 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1856e-05 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957294+0000", 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5027e-05 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5972 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.156280344999999, 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043278321000000002, 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5972 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.914192+0000", 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915928+0000", 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001735903 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915932+0000", 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915940+0000", 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7390000000000005e-06 2024-09-15T00:11:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.915947+0000", 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.567e-06 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957328+0000", 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041381322999999998 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957417+0000", 2024-09-15T00:11:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.957470+0000", 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3263000000000001e-05 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5981 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4444160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.154416689000001, 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042536860000000003, 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5981 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916055+0000", 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916056+0000", 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1400000000000001e-07 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916057+0000", 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7399999999999991e-07 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916060+0000", 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4939999999999999e-06 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916063+0000", 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6989999999999999e-06 2024-09-15T00:11:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958500+0000", 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042437366999999997 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958542+0000", 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2004e-05 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958592+0000", 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0108000000000002e-05 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5982 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4485120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.15438692, 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042623855000000002, 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5982 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916085+0000", 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916086+0000", 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-07 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916088+0000", 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916093+0000", 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916100+0000", 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7519999999999999e-06 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958617+0000", 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042516801 2024-09-15T00:11:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958660+0000", 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3618999999999997e-05 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958709+0000", 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8634000000000002e-05 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5983 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4526080~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.154364741000002, 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042717786000000001, 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5983 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916107+0000", 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916108+0000", 2024-09-15T00:11:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1399999999999998e-07 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916110+0000", 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9609999999999999e-06 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916119+0000", 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6439999999999995e-06 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916127+0000", 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6490000000000008e-06 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958734+0000", 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042607196999999999 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958776+0000", 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1984000000000003e-05 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958825+0000", 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9036999999999998e-05 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5984 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4567040~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.154337491, 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042811210000000002, 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5984 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916135+0000", 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4900000000000001e-07 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916137+0000", 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.029e-06 2024-09-15T00:11:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916143+0000", 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8819999999999998e-06 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916152+0000", 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0780000000000002e-06 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958850+0000", 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042697865000000002 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958893+0000", 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3392000000000001e-05 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958946+0000", 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2614999999999997e-05 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5985 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4608000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.154310897000002, 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042900122999999998, 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5985 2024-09-15T00:11:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916161+0000", 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916162+0000", 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000001e-07 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916164+0000", 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9929999999999998e-06 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916169+0000", 2024-09-15T00:11:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8760000000000002e-06 2024-09-15T00:11:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916174+0000", 2024-09-15T00:11:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8230000000000002e-06 2024-09-15T00:11:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.958970+0000", 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042796169000000002 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959012+0000", 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1347999999999998e-05 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959061+0000", 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9527000000000002e-05 2024-09-15T00:11:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5986 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 4648960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.154291563999998, 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043138487000000003, 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5986 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.916181+0000", 2024-09-15T00:11:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917929+0000", 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001748105 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917932+0000", 2024-09-15T00:11:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1769999999999998e-06 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917938+0000", 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5860000000000001e-06 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.917943+0000", 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3370000000000001e-06 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959085+0000", 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041142004000000003 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959128+0000", 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3052e-05 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.959319+0000", 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019122599999999999 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5998 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5140480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.152243155000001, 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042508254000000002, 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5998 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918229+0000", 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0900000000000001e-07 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918230+0000", 2024-09-15T00:11:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.204e-06 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918235+0000", 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689999999999997e-06 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918238+0000", 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960645+0000", 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042407322999999997 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960687+0000", 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2305000000000002e-05 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960737+0000", 2024-09-15T00:11:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0039e-05 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5568.0:5999 3.14 3:2fee524d:::rbd_data.1129696a4676.0000000000000003:24 [read 5181440~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.152227922000002, 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.042609460000000002, 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5568", 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3960523877", 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5999 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.918244+0000", 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919935+0000", 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690907 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919938+0000", 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2949999999999998e-06 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919945+0000", 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.691e-06 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.919949+0000", 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0539999999999996e-06 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960762+0000", 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.040812637999999998 2024-09-15T00:11:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960805+0000", 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3238000000000002e-05 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:14.960854+0000", 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8637000000000002e-05 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:674 3.12 3:4cd1bb64:::rbd_data.1129466cbf24.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.386951402999998, 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.067426725000000007, 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 674 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683521+0000", 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683523+0000", 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729458+0000", 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045934807000000001 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729484+0000", 2024-09-15T00:11:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6200999999999999e-05 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729494+0000", 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0015e-05 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729566+0000", 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1644999999999996e-05 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729715+0000", 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000149276 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750792+0000", 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021076527000000001 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750836+0000", 2024-09-15T00:11:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4193000000000002e-05 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.750947+0000", 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111661 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:682 3.15 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18087936~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.340860733, 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044589076999999998, 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 682 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729611+0000", 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.729612+0000", 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8999999999999996e-07 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760825+0000", 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.03121345 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760853+0000", 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7416e-05 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760863+0000", 2024-09-15T00:11:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0402e-05 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.760941+0000", 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7900999999999998e-05 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.761065+0000", 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012336399999999999 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774084+0000", 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013019307000000001 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774126+0000", 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2401000000000003e-05 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.774200+0000", 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4146000000000005e-05 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.172 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.172 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.172 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:14.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:14.174 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:14.171+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:14.273 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:14.272+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:14.317 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-15T00:11:14.317 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:14.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:14.374 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:14.373+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:14.467 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:14.468 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:14.475 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:14.473+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:14.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:14.576 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:14.575+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:14.620 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:14.620 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:14.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:14.620 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:14.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.621 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1143 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.621 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:14.621 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.735189041, 2024-09-15T00:11:14.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045317713000000003, 2024-09-15T00:11:14.621 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.621 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1143 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882804+0000", 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882805+0000", 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9500000000000002e-07 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882813+0000", 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0099999999999995e-06 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882821+0000", 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7229999999999993e-06 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882833+0000", 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226000000000001e-05 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.927975+0000", 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045142822999999999 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928029+0000", 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3340000000000001e-05 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928122+0000", 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2996000000000002e-05 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1144 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73515041100001, 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457639000000001, 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.626 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1144 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882843+0000", 2024-09-15T00:11:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4799999999999998e-07 2024-09-15T00:11:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882846+0000", 2024-09-15T00:11:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5399999999999998e-06 2024-09-15T00:11:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882853+0000", 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882858+0000", 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9629999999999997e-06 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928151+0000", 2024-09-15T00:11:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045293896 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928209+0000", 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7596999999999998e-05 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928300+0000", 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1168000000000005e-05 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1145 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 630784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.735124474, 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045619767999999998, 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1145 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882869+0000", 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-07 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882870+0000", 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4959999999999999e-06 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882875+0000", 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8679999999999998e-06 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882878+0000", 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380000000000001e-06 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928331+0000", 2024-09-15T00:11:14.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045452962999999999 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928396+0000", 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4879e-05 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928488+0000", 2024-09-15T00:11:14.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2348000000000004e-05 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1146 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73510840599999, 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045798195999999999, 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.636 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1146 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:14.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882885+0000", 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882992+0000", 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107277 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882994+0000", 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4619999999999999e-06 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.882999+0000", 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6129999999999997e-06 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883003+0000", 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5580000000000001e-06 2024-09-15T00:11:14.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928518+0000", 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045515235000000001 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928591+0000", 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3493999999999998e-05 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928683+0000", 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1557000000000004e-05 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1147 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73496731100001, 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045795938000000001, 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.641 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1147 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883026+0000", 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883733+0000", 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000707017 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883737+0000", 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5020000000000003e-06 2024-09-15T00:11:14.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883760+0000", 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3150999999999999e-05 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883774+0000", 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4018e-05 2024-09-15T00:11:14.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928719+0000", 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044944785000000001 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928770+0000", 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0594999999999999e-05 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.928822+0000", 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1870000000000003e-05 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1149 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73411052500001, 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045260858000000001, 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.646 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1149 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883882+0000", 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883883+0000", 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7999999999999995e-07 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883886+0000", 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883903+0000", 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7263000000000001e-05 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883909+0000", 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8869999999999997e-06 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929029+0000", 2024-09-15T00:11:14.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045120423 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929087+0000", 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.804e-05 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929143+0000", 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6100999999999997e-05 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1150 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73407550899999, 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045350635, 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.651 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1150 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883917+0000", 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883918+0000", 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7100000000000002e-07 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883920+0000", 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2469999999999998e-06 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883931+0000", 2024-09-15T00:11:14.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1009000000000001e-05 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883936+0000", 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1039999999999998e-06 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929168+0000", 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232133000000001 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929219+0000", 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0865000000000002e-05 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929268+0000", 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8806e-05 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1151 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.655 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.734049906, 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045457338999999999, 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1151 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.883943+0000", 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884088+0000", 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145148 2024-09-15T00:11:14.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884092+0000", 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.087e-06 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884101+0000", 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884107+0000", 2024-09-15T00:11:14.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9660000000000001e-06 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929297+0000", 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045189372999999998 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929349+0000", 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2166999999999997e-05 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929400+0000", 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1637e-05 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1152 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73387563999999, 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045404683000000001, 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.660 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1152 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884117+0000", 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884313+0000", 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000195317 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884317+0000", 2024-09-15T00:11:14.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.673e-06 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884327+0000", 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3929999999999997e-06 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884336+0000", 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9819999999999997e-06 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929432+0000", 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045096217000000001 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929477+0000", 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5534000000000001e-05 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929522+0000", 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4567000000000002e-05 2024-09-15T00:11:14.664 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1153 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.733644501, 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045315448000000001, 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.665 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1153 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884349+0000", 2024-09-15T00:11:14.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884990+0000", 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064183700000000003 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.884995+0000", 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6099999999999999e-06 2024-09-15T00:11:14.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885008+0000", 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2621e-05 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885014+0000", 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6239999999999996e-06 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929549+0000", 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044534408999999997 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929585+0000", 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6832999999999998e-05 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.929664+0000", 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8514000000000001e-05 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.669 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1158 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73283255699999, 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045280768999999998, 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.670 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1158 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885160+0000", 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885214+0000", 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3798000000000002e-05 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885217+0000", 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2440000000000002e-06 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885226+0000", 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3089999999999996e-06 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885231+0000", 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5670000000000002e-06 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930342+0000", 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045110737999999997 2024-09-15T00:11:14.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930379+0000", 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7376999999999998e-05 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930441+0000", 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1736000000000007e-05 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1159 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73275281700001, 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045400888, 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.675 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1159 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885240+0000", 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885417+0000", 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177064 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885421+0000", 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3340000000000002e-06 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885428+0000", 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5460000000000003e-06 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885435+0000", 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.204e-06 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930481+0000", 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045045229999999999 2024-09-15T00:11:14.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930557+0000", 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6352000000000004e-05 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930641+0000", 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4158000000000003e-05 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1160 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73254779200001, 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045319348000000002, 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1160 2024-09-15T00:11:14.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885445+0000", 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885634+0000", 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018892799999999999 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885637+0000", 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2380000000000002e-06 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885644+0000", 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9140000000000002e-06 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885649+0000", 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000003e-06 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930669+0000", 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045020154 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930706+0000", 2024-09-15T00:11:14.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7107000000000002e-05 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930765+0000", 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8087000000000001e-05 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1161 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:14.684 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.73233625899999, 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045241585000000001, 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1161 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885657+0000", 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885931+0000", 2024-09-15T00:11:14.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027401599999999997 2024-09-15T00:11:14.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885935+0000", 2024-09-15T00:11:14.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8809999999999998e-06 2024-09-15T00:11:14.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885942+0000", 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4970000000000004e-06 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.885948+0000", 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930807+0000", 2024-09-15T00:11:14.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044859647000000002 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930847+0000", 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9950000000000002e-05 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.930898+0000", 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0979000000000001e-05 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.689 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1174 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 2531328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.729780252, 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045232887999999999, 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1174 2024-09-15T00:11:14.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.888213+0000", 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889930+0000", 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017168800000000001 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889934+0000", 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7729999999999999e-06 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889945+0000", 2024-09-15T00:11:14.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0557000000000001e-05 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.889964+0000", 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8584e-05 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933100+0000", 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043136358 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933170+0000", 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0512000000000005e-05 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.933446+0000", 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027522400000000002 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1185 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.694 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.727797146, 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045218019999999998, 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890196+0000", 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-15T00:11:14.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890198+0000", 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890204+0000", 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.319e-06 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890209+0000", 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0899999999999998e-06 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935269+0000", 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045060595000000002 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935341+0000", 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1721e-05 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935414+0000", 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3035999999999997e-05 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4731.0:1186 3.c 3:34ae4a59:::rbd_data.112910301a74.000000000000000f:b [read 3317760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.727777575, 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045310982, 2024-09-15T00:11:14.699 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4731", 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3926520721", 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1186 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.890215+0000", 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891931+0000", 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001715945 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891936+0000", 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.656e-06 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891947+0000", 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1391e-05 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.891954+0000", 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.934e-06 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935444+0000", 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043490124999999998 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935487+0000", 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2391999999999997e-05 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:09:16.935526+0000", 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9539000000000002e-05 2024-09-15T00:11:14.703 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5072.0:11361 3.17 3:ec331c6b:::rbd_data.1129203d0835.000000000000000b:1c [read 782336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-15T00:11:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 72.457034725, 2024-09-15T00:11:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045429809000000002, 2024-09-15T00:11:14.704 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5072", 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/3800782018", 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 11361 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160958+0000", 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160959+0000", 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160960+0000", 2024-09-15T00:11:14.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.575e-06 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160965+0000", 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1250000000000001e-06 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.160970+0000", 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.228e-06 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206196+0000", 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045226810999999999 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206276+0000", 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079000000000006e-05 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:02.206388+0000", 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112586 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.708 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:18853 3.f 3:f5b607ae:::rbd_header.11297ef179c5:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.551283230000003, 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047340943000000003, 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:14.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18853 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.066710+0000", 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.112995+0000", 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046285639000000003 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113000+0000", 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0520000000000001e-06 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113007+0000", 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7100000000000001e-06 2024-09-15T00:11:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113012+0000", 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1950000000000002e-06 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113937+0000", 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092437400000000005 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.113997+0000", 2024-09-15T00:11:14.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0395e-05 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:15.114051+0000", 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3578000000000001e-05 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.5548.0:676 3.17 3:ed0a8897:::rbd_data.1129466cbf24.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18350080~262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:14.713 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.934195789999997, 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073335735999999999, 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.5548", 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/277363372", 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:14.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683797+0000", 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.683799+0000", 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6190000000000001e-06 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733010+0000", 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.049211194 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733043+0000", 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2802999999999997e-05 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733057+0000", 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4185999999999999e-05 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733128+0000", 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0875000000000002e-05 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.733242+0000", 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114407 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.756907+0000", 2024-09-15T00:11:14.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023664339 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757005+0000", 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8523000000000005e-05 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:18.757133+0000", 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012778999999999999 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.719 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.719 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:14.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:14.721 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:14.718+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:14.814 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T00:11:14.820 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:14.819+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:14.875 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:28404 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-15T00:11:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:14.874145+0000", 2024-09-15T00:11:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00042739200000000002, 2024-09-15T00:11:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00045143400000000001, 2024-09-15T00:11:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 28404 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:14.874145+0000", 2024-09-15T00:11:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:14.874145+0000", 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:14.874145+0000", 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8199999999999999e-07 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:14.874150+0000", 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5079999999999999e-06 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:14.874160+0000", 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6730000000000004e-06 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:14.874165+0000", 2024-09-15T00:11:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9219999999999996e-06 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:14.874204+0000", 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9274999999999999e-05 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:14.874254+0000", 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0090999999999999e-05 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:14.874368+0000", 2024-09-15T00:11:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114518 2024-09-15T00:11:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-15T00:11:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-15T00:11:14.881 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:14.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:14.921 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:14.920+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:14.985 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T00:11:14.994 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:14.995 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-15T00:11:14.995 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-15T00:11:14.995 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-15T00:11:14.995 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:15.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:15.030 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:15.029+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:15.108 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-15T00:11:15.108 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-15T00:11:15.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-15T00:11:15.108 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-15T00:11:15.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.108 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1893 3.1e e119/118 3:7c90e45f:::rbd_data.11291f9c3154.000000000000003a:1d v 119'463, mlcod=119'459)", 2024-09-15T00:11:15.108 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:15.108 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.676883547999999, 2024-09-15T00:11:15.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005948208, 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:15.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428514+0000", 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.37e-07 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428523+0000", 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2139999999999996e-06 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428532+0000", 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7229999999999999e-06 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428539+0000", 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3289999999999999e-06 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428578+0000", 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8792999999999999e-05 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.428595+0000", 2024-09-15T00:11:15.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7669e-05 2024-09-15T00:11:15.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.434405+0000", 2024-09-15T00:11:15.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058096780000000004 2024-09-15T00:11:15.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.434462+0000", 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7065000000000003e-05 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3933 3.15 e119/118 3:ad39a9e7:::rbd_data.11291d0c7d7c.0000000000000000:16 v 119'404, mlcod=115'401)", 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.675559229000001, 2024-09-15T00:11:15.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075329059999999998, 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429838+0000", 2024-09-15T00:11:15.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429839+0000", 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5799999999999999e-07 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429844+0000", 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7649999999999999e-06 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429855+0000", 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0644e-05 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429862+0000", 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.477e-06 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429903+0000", 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1693999999999998e-05 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.429931+0000", 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7699e-05 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.437329+0000", 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073982270000000003 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.437371+0000", 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1742000000000001e-05 2024-09-15T00:11:15.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1895 3.1e e119/118 3:7d7cb6ca:::rbd_data.11294a896ed0.0000000000000010:22 v 119'467, mlcod=119'463)", 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.665061193, 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007846713, 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440336+0000", 2024-09-15T00:11:15.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440341+0000", 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7350000000000004e-06 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440349+0000", 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6299999999999998e-06 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440357+0000", 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1710000000000002e-06 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440413+0000", 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5980000000000003e-05 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440432+0000", 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8847000000000001e-05 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.448132+0000", 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007700386 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.448183+0000", 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0559000000000002e-05 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3934 3.15 e119/118 3:a9ed06fd:::rbd_data.112986cdeff1.0000000000000014:1b v 119'406, mlcod=116'402)", 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.664578299999999, 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010330291, 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440819+0000", 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440820+0000", 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3099999999999997e-07 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440825+0000", 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.186e-06 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440836+0000", 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.092e-05 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440842+0000", 2024-09-15T00:11:15.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3890000000000002e-06 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440903+0000", 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1393e-05 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.440921+0000", 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7975e-05 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451109+0000", 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010187391000000001 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451149+0000", 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0405999999999999e-05 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:1897 3.1e e119/118 3:7a2239c1:::rbd_data.1129466cbf24.000000000000000a:27 v 119'471, mlcod=119'465)", 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:15.127 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.654414069000001, 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005507311, 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450983+0000", 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450984+0000", 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450987+0000", 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9409999999999999e-06 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450992+0000", 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2789999999999998e-06 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.450995+0000", 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0740000000000001e-06 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451734+0000", 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073883599999999998 2024-09-15T00:11:15.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.451756+0000", 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2158999999999999e-05 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.456450+0000", 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004693666 2024-09-15T00:11:15.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.456491+0000", 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0976000000000002e-05 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3936 3.15 e119/118 3:a9d1a5ec:::rbd_data.1129696a4676.0000000000000006:24 v 119'410, mlcod=119'406)", 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.643766905, 2024-09-15T00:11:15.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006441413, 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461630+0000", 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461631+0000", 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4899999999999995e-07 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461636+0000", 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6659999999999999e-06 2024-09-15T00:11:15.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461644+0000", 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5269999999999998e-06 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461653+0000", 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5620000000000004e-06 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461705+0000", 2024-09-15T00:11:15.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2045000000000001e-05 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.461724+0000", 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.893e-05 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.468031+0000", 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006307773 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.468072+0000", 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0361000000000002e-05 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3937 3.15 e119/118 3:a96659a7:::rbd_data.1129466cbf24.0000000000000008:27 v 119'412, mlcod=119'408)", 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.631196715000002, 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077903419999999996, 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.137 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474201+0000", 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1699999999999997e-07 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474207+0000", 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0820000000000004e-06 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474218+0000", 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.111e-05 2024-09-15T00:11:15.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474225+0000", 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7830000000000001e-06 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474267+0000", 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1644999999999999e-05 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.474288+0000", 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0693999999999999e-05 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.481944+0000", 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076562280000000002 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.481991+0000", 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7083000000000003e-05 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5281 3.e e119/118 3:771a58a1:::rbd_data.1129eb1f5358.0000000000000007:18 v 119'285, mlcod=110'280)", 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.618879642, 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064605180000000002, 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:15.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:15.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486518+0000", 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1900000000000003e-07 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486523+0000", 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2159999999999997e-06 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486531+0000", 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2570000000000001e-06 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486535+0000", 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4170000000000002e-06 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486578+0000", 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099000000000002e-05 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.486599+0000", 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0735999999999999e-05 2024-09-15T00:11:15.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.492941+0000", 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063422289999999996 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.492978+0000", 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7045000000000002e-05 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.5.0:5283 3.e e119/118 3:7642e0f1:::rbd_data.112929d537d9.000000000000002c:1e v 119'289, mlcod=119'283)", 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.608177061999999, 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064221740000000001, 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497220+0000", 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497221+0000", 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7299999999999995e-07 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497226+0000", 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0239999999999997e-06 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497238+0000", 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1746000000000001e-05 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497244+0000", 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6640000000000001e-06 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497287+0000", 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3130999999999998e-05 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.497307+0000", 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9740000000000001e-05 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.503604+0000", 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062964780000000003 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.503642+0000", 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8717999999999998e-05 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:3943 3.14 e119/118 3:2a8444a6:::rbd_data.11291d0c7d7c.0000000000000003:16 v 119'548, mlcod=116'544)", 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.508584717000002, 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064816789999999997, 2024-09-15T00:11:15.151 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596813+0000", 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6300000000000005e-07 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596819+0000", 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8189999999999997e-06 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596827+0000", 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1200000000000002e-06 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596832+0000", 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5179999999999998e-06 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596870+0000", 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.803e-05 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.596891+0000", 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1746000000000002e-05 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.603253+0000", 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006361663 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:10:58.603294+0000", 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1220000000000002e-05 2024-09-15T00:11:15.155 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:25995 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.678306389999999, 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.948917703, 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:15.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 25995 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427091+0000", 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427092+0000", 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.48e-06 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427100+0000", 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.072e-06 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427113+0000", 2024-09-15T00:11:15.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2845e-05 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427124+0000", 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0324e-05 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427177+0000", 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3514999999999998e-05 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:00.427184+0000", 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1910000000000003e-06 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.374306+0000", 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.947121671 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.374394+0000", 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7918999999999996e-05 2024-09-15T00:11:15.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.374538+0000", 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014439100000000001 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.375182+0000", 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064394599999999995 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.375954+0000", 2024-09-15T00:11:15.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077125800000000003 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.375985+0000", 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1483999999999999e-05 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:01.376009+0000", 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3606999999999999e-05 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:27417 3.1a 3:5f9784b4:::rbd_data.112910301a74.0000000000000003:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:09.500472+0000", 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.6049249940000001, 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052738500000000001, 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 27417 2024-09-15T00:11:15.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500472+0000", 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500472+0000", 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500474+0000", 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3850000000000001e-06 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500480+0000", 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2600000000000002e-06 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500495+0000", 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4908e-05 2024-09-15T00:11:15.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500504+0000", 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1870000000000003e-06 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500565+0000", 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0792999999999999e-05 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500858+0000", 2024-09-15T00:11:15.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029332000000000002 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500961+0000", 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103322 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.502687+0000", 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001725589 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505688+0000", 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030005980000000002 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505719+0000", 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1627000000000001e-05 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505746+0000", 2024-09-15T00:11:15.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6860999999999999e-05 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:27419 3.9 3:926cd00d:::rbd_data.112910301a74.0000000000000005:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:09.500527+0000", 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.6048698889999997, 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052930809999999998, 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.169 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 27419 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500527+0000", 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500527+0000", 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500528+0000", 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0100000000000005e-07 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500531+0000", 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7870000000000002e-06 2024-09-15T00:11:15.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500539+0000", 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8960000000000003e-06 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500545+0000", 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9909999999999999e-06 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500644+0000", 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9451999999999996e-05 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.500942+0000", 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029790399999999999 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.501476+0000", 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053428700000000004 2024-09-15T00:11:15.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.502813+0000", 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001337038 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505729+0000", 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029160050000000002 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505782+0000", 2024-09-15T00:11:15.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2472000000000002e-05 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:09.505820+0000", 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8748000000000002e-05 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:27482 3.1 3:8503df82:::rbd_object_map.1129b452887f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:10.429608+0000", 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6757894440000003, 2024-09-15T00:11:15.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061838010000000001, 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 27482 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.429608+0000", 2024-09-15T00:11:15.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.429608+0000", 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.429609+0000", 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5400000000000001e-06 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.429616+0000", 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4710000000000004e-06 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.429627+0000", 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1406000000000001e-05 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.429632+0000", 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1590000000000001e-06 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.429699+0000", 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6278999999999993e-05 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.429705+0000", 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.392e-06 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.430314+0000", 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00060875800000000004 2024-09-15T00:11:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.430400+0000", 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6487000000000001e-05 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.430587+0000", 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000186195 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.434125+0000", 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035381430000000001 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.435683+0000", 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015583890000000001 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.435741+0000", 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8236000000000003e-05 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.435792+0000", 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0346000000000002e-05 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:27579 3.9 3:9200da80:::rbd_data.1129b452887f.000000000000000d:head [delete] snapc a=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:10.509265+0000", 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.5961320700000003, 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058510000000000003, 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 27579 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.509265+0000", 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.509265+0000", 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.509266+0000", 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.147e-06 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.509274+0000", 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9899999999999997e-06 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.509287+0000", 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2496e-05 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.509294+0000", 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1640000000000004e-06 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.509987+0000", 2024-09-15T00:11:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00069268499999999996 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.510175+0000", 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018874300000000001 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.510288+0000", 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112321 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.511734+0000", 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014460670000000001 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.515026+0000", 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032917649999999999 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.515079+0000", 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3531e-05 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:10.515116+0000", 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7091000000000001e-05 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:28248 3.1 3:817244d0:::rbd_data.11291ce3fc54.0000000000000005:head [delete] snapc 6=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:13.909712+0000", 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1956855740000001, 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068464709999999998, 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 28248 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909712+0000", 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909712+0000", 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909713+0000", 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1990000000000001e-06 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909718+0000", 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.028e-06 2024-09-15T00:11:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909731+0000", 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3084e-05 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909737+0000", 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0050000000000001e-06 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909799+0000", 2024-09-15T00:11:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1866999999999996e-05 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910067+0000", 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026797099999999999 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910187+0000", 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120133 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.912954+0000", 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027666600000000002 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.916460+0000", 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035058849999999998 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.916514+0000", 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4595000000000002e-05 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.916558+0000", 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4044e-05 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:28251 3.1b 3:dd86e936:::rbd_data.11291ce3fc54.0000000000000008:head [delete] snapc 6=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:13.909758+0000", 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.195638899, 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070944490000000001, 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 28251 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909758+0000", 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909758+0000", 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909759+0000", 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.34e-07 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909762+0000", 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9890000000000001e-06 2024-09-15T00:11:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909769+0000", 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9110000000000004e-06 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909774+0000", 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0379999999999999e-06 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.909834+0000", 2024-09-15T00:11:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0522000000000001e-05 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910068+0000", 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023334399999999999 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910175+0000", 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107354 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.911149+0000", 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097452200000000002 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.916755+0000", 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056050759999999996 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.916810+0000", 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5758999999999999e-05 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.916853+0000", 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2500000000000003e-05 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4393.0:28250 3.1e e135/118 3:7c6288f2:::rbd_data.11291ce3fc54.0000000000000007:head v 135'496, mlcod=134'493)", 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:13.910834+0000", 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1945637689999999, 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005676464, 2024-09-15T00:11:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910834+0000", 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910834+0000", 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910834+0000", 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8699999999999995e-07 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910839+0000", 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6530000000000002e-06 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910850+0000", 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0878e-05 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910855+0000", 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8270000000000003e-06 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910919+0000", 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3159999999999998e-05 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.910947+0000", 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8102e-05 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.916468+0000", 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055215780000000001 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.916510+0000", 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1678999999999999e-05 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:28259 3.1 3:85c6cf79:::rbd_data.11291ce3fc54.0000000000000011:head [delete] snapc 6=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:13.913098+0000", 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.192299177, 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061374029999999996, 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 28259 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913098+0000", 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913098+0000", 2024-09-15T00:11:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913099+0000", 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.297e-06 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913107+0000", 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4590000000000001e-06 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913120+0000", 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2833999999999999e-05 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913126+0000", 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7739999999999999e-06 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913176+0000", 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9234000000000003e-05 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913319+0000", 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014331399999999999 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913377+0000", 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8072999999999998e-05 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.914247+0000", 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086950899999999997 2024-09-15T00:11:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.919165+0000", 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049179519999999997 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.919203+0000", 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8787e-05 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.919236+0000", 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2169999999999999e-05 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4393.0:28260 3.1a 3:5e7c420a:::rbd_data.11291ce3fc54.0000000000000012:head [delete] snapc 6=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-15T00:11:13.913146+0000", 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1922507929999999, 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060863990000000001, 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "continuous": false, 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T00:11:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4393", 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.92:0/2763983199", 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 28260 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913146+0000", 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913146+0000", 2024-09-15T00:11:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913147+0000", 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9599999999999999e-07 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913150+0000", 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6599999999999999e-06 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913156+0000", 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9270000000000002e-06 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913159+0000", 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.681e-06 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913195+0000", 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5255e-05 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913313+0000", 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118631 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.913375+0000", 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1767999999999995e-05 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.914304+0000", 2024-09-15T00:11:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092887799999999995 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.919144+0000", 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048405760000000001 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.919195+0000", 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0166e-05 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-15T00:11:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-15T00:11:13.919233+0000", 2024-09-15T00:11:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8361000000000001e-05 2024-09-15T00:11:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-15T00:11:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-15T00:11:15.217 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-15T00:11:15.218 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T00:11:15.218 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T00:11:15.219 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T00:11:15.219 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T00:11:15.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T00:11:15.220 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T00:11:15.220 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T00:11:15.221 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T00:11:15.221 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T00:11:15.221 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T00:11:15.221 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T00:11:15.222 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T00:11:15.222 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T00:11:15.223 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:15.217+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:15.319 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:15.317+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:15.369 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:15.370 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:15.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T00:11:15.420 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:15.418+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:15.485 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:15.485 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:15.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T00:11:15.521 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:15.519+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:15.600 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:15.600 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:15.609 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T00:11:15.653 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:15.652+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:15.716 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:15.716 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:15.723 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:15.721+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:15.725 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T00:11:15.824 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:15.823+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:15.878 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:15.878 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:15.885 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:15.887 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T00:11:15.930 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:15.928+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:16.026 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:16.025+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:16.039 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:16.039 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:16.039 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:16.039 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:16.049 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T00:11:16.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T00:11:16.133 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T00:11:16.132+0000 7f1c19bc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 55387) UID: 0 2024-09-15T00:11:16.153 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:16.153 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:16.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T00:11:16.246 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:16.244+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:16.346 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:16.345+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:16.347 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:16.347 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:16.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T00:11:16.447 INFO:tasks.ceph.osd.5.smithi079.stderr:2024-09-15T00:11:16.446+0000 7f96f0969640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32257) UID: 0 2024-09-15T00:11:16.500 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:16.500 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:16.509 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T00:11:16.548 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T00:11:16.547+0000 7f6fb0668640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31787) UID: 0 2024-09-15T00:11:16.649 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-15T00:11:16.647+0000 7f6e9a9b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32252) UID: 0 2024-09-15T00:11:16.651 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:16.651 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:16.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T00:11:16.749 INFO:tasks.ceph.osd.4.smithi079.stderr:2024-09-15T00:11:16.748+0000 7ff914f13640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43643) UID: 0 2024-09-15T00:11:16.802 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:16.802 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T00:11:16.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T00:11:16.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T00:11:16.834 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:11:16.867 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T00:11:16.866+0000 7f2a3f981640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31790) UID: 0 2024-09-15T00:11:16.920 INFO:teuthology.orchestra.run.smithi092.stdout:All operations completed A-OK! 2024-09-15T00:11:16.930 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:16.931 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-15T00:11:16.986 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-15T00:11:16.996 INFO:tasks.thrashosds:joining thrashosds 2024-09-15T00:11:16.998 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:16.998 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:16.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T00:11:17.102 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:17.102 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:17.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T00:11:17.215 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:17.215 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:17.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:17.326 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:17.336 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T00:11:17.444 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:17.444 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:17.453 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T00:11:17.560 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:17.560 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:17.569 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T00:11:17.676 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T00:11:17.676 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T00:11:17.826 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T00:11:17.827 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:11:17.841 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T00:11:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T00:11:18.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T00:11:19.835 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T00:11:19.842 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T00:11:19.850 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T00:11:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T00:11:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T00:11:20.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T00:11:20.864 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T00:11:20.864 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:21.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:21.220 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:11:21.232 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-15T00:11:20.878938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":79,"num_read":254996,"num_read_kb":1398234,"num_write":21895,"num_write_kb":430524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1263,"num_bytes_recovered":92864904,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19789,"ondisk_log_size":19789,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":266728,"kb_used_data":8220,"kb_used_omap":37,"kb_used_meta":258330,"kb_avail":471592472,"statfs":{"total":483183820800,"available":482910691328,"internally_reserved":0,"allocated":8417280,"data_stored":4912102,"data_compressed":19298,"data_compressed_allocated":102400,"data_compressed_original":1400832,"omap_allocated":38132,"internal_metadata":264530700},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-41133580,"num_objects":-143,"num_object_clones":-41,"num_object_copies":-286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-143,"num_whiteouts":-35,"num_read":853,"num_read_kb":723,"num_write":185,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000884"},"pg_stats":[{"pgid":"3.1f","version":"138'535","reported_seq":11553,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.822559+0000","last_change":"2024-09-15T00:11:16.822559+0000","last_active":"2024-09-15T00:11:16.822559+0000","last_peered":"2024-09-15T00:11:16.822559+0000","last_clean":"2024-09-15T00:11:16.822559+0000","last_became_active":"2024-09-15T00:10:26.157785+0000","last_became_peered":"2024-09-15T00:10:26.157785+0000","last_unstale":"2024-09-15T00:11:16.822559+0000","last_undegraded":"2024-09-15T00:11:16.822559+0000","last_fullsized":"2024-09-15T00:11:16.822559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:45:18.226279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001206129,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12081,"num_read_kb":64593,"num_write":535,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1e","version":"135'496","reported_seq":5487,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825201+0000","last_change":"2024-09-15T00:11:16.808902+0000","last_active":"2024-09-15T00:11:17.825201+0000","last_peered":"2024-09-15T00:11:17.825201+0000","last_clean":"2024-09-15T00:11:17.825201+0000","last_became_active":"2024-09-15T00:10:58.347628+0000","last_became_peered":"2024-09-15T00:10:58.347628+0000","last_unstale":"2024-09-15T00:11:17.825201+0000","last_undegraded":"2024-09-15T00:11:17.825201+0000","last_fullsized":"2024-09-15T00:11:17.825201+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:19.680950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014193910000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6067,"num_read_kb":40333,"num_write":376,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1d","version":"137'826","reported_seq":5668,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825771+0000","last_change":"2024-09-15T00:11:16.819173+0000","last_active":"2024-09-15T00:11:17.825771+0000","last_peered":"2024-09-15T00:11:17.825771+0000","last_clean":"2024-09-15T00:11:17.825771+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:11:17.825771+0000","last_undegraded":"2024-09-15T00:11:17.825771+0000","last_fullsized":"2024-09-15T00:11:17.825771+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:38.089567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019789569999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8748,"num_read_kb":37428,"num_write":973,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1c","version":"138'512","reported_seq":6088,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825601+0000","last_change":"2024-09-15T00:11:16.818860+0000","last_active":"2024-09-15T00:11:17.825601+0000","last_peered":"2024-09-15T00:11:17.825601+0000","last_clean":"2024-09-15T00:11:17.825601+0000","last_became_active":"2024-09-15T00:10:58.347433+0000","last_became_peered":"2024-09-15T00:10:58.347433+0000","last_unstale":"2024-09-15T00:11:17.825601+0000","last_undegraded":"2024-09-15T00:11:17.825601+0000","last_fullsized":"2024-09-15T00:11:17.825601+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:34.872358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878240000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6371,"num_read_kb":39648,"num_write":614,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1b","version":"137'519","reported_seq":10455,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.835389+0000","last_change":"2024-09-15T00:11:15.835389+0000","last_active":"2024-09-15T00:11:15.835389+0000","last_peered":"2024-09-15T00:11:15.835389+0000","last_clean":"2024-09-15T00:11:15.835389+0000","last_became_active":"2024-09-15T00:10:58.352065+0000","last_became_peered":"2024-09-15T00:10:58.352065+0000","last_unstale":"2024-09-15T00:11:15.835389+0000","last_undegraded":"2024-09-15T00:11:15.835389+0000","last_fullsized":"2024-09-15T00:11:15.835389+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:06:28.305710+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016246954000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11088,"num_read_kb":77879,"num_write":574,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"3.1a","version":"137'534","reported_seq":11068,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.961475+0000","last_change":"2024-09-15T00:11:15.826478+0000","last_active":"2024-09-15T00:11:15.961475+0000","last_peered":"2024-09-15T00:11:15.961475+0000","last_clean":"2024-09-15T00:11:15.961475+0000","last_became_active":"2024-09-15T00:10:58.346770+0000","last_became_peered":"2024-09-15T00:10:58.346770+0000","last_unstale":"2024-09-15T00:11:15.961475+0000","last_undegraded":"2024-09-15T00:11:15.961475+0000","last_fullsized":"2024-09-15T00:11:15.961475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:03.547839+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0073748930000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":13906,"num_read_kb":40584,"num_write":444,"num_write_kb":8754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"3.19","version":"136'709","reported_seq":10753,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.810826+0000","last_change":"2024-09-15T00:11:16.811299+0000","last_active":"2024-09-15T00:11:16.810826+0000","last_peered":"2024-09-15T00:11:16.810826+0000","last_clean":"2024-09-15T00:11:16.810826+0000","last_became_active":"2024-09-15T00:10:58.350007+0000","last_became_peered":"2024-09-15T00:10:58.350007+0000","last_unstale":"2024-09-15T00:11:16.810826+0000","last_undegraded":"2024-09-15T00:11:16.810826+0000","last_fullsized":"2024-09-15T00:11:16.810826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:11.344098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019477310000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13597,"num_read_kb":55399,"num_write":535,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.18","version":"137'655","reported_seq":8623,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.824975+0000","last_change":"2024-09-15T00:11:16.808297+0000","last_active":"2024-09-15T00:11:17.824975+0000","last_peered":"2024-09-15T00:11:17.824975+0000","last_clean":"2024-09-15T00:11:17.824975+0000","last_became_active":"2024-09-15T00:10:26.158999+0000","last_became_peered":"2024-09-15T00:10:26.158999+0000","last_unstale":"2024-09-15T00:11:17.824975+0000","last_undegraded":"2024-09-15T00:11:17.824975+0000","last_fullsized":"2024-09-15T00:11:17.824975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:11:07.970440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012280349999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9415,"num_read_kb":64591,"num_write":699,"num_write_kb":14239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.17","version":"136'859","reported_seq":15029,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825763+0000","last_change":"2024-09-15T00:11:16.818448+0000","last_active":"2024-09-15T00:11:17.825763+0000","last_peered":"2024-09-15T00:11:17.825763+0000","last_clean":"2024-09-15T00:11:17.825763+0000","last_became_active":"2024-09-15T00:10:26.158796+0000","last_became_peered":"2024-09-15T00:10:26.158796+0000","last_unstale":"2024-09-15T00:11:17.825763+0000","last_undegraded":"2024-09-15T00:11:17.825763+0000","last_fullsized":"2024-09-15T00:11:17.825763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:27.044437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001285972,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":16900,"num_read_kb":72084,"num_write":902,"num_write_kb":44641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.16","version":"135'832","reported_seq":8136,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825736+0000","last_change":"2024-09-15T00:11:16.818288+0000","last_active":"2024-09-15T00:11:17.825736+0000","last_peered":"2024-09-15T00:11:17.825736+0000","last_clean":"2024-09-15T00:11:17.825736+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:11:17.825736+0000","last_undegraded":"2024-09-15T00:11:17.825736+0000","last_fullsized":"2024-09-15T00:11:17.825736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.421381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098720300000000008,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10216,"num_read_kb":33960,"num_write":980,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.15","version":"136'460","reported_seq":6838,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820887+0000","last_change":"2024-09-15T00:11:16.821358+0000","last_active":"2024-09-15T00:11:16.820887+0000","last_peered":"2024-09-15T00:11:16.820887+0000","last_clean":"2024-09-15T00:11:16.820887+0000","last_became_active":"2024-09-15T00:10:58.351863+0000","last_became_peered":"2024-09-15T00:10:58.351863+0000","last_unstale":"2024-09-15T00:11:16.820887+0000","last_undegraded":"2024-09-15T00:11:16.820887+0000","last_fullsized":"2024-09-15T00:11:16.820887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:36:31.765520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081705799999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6309,"num_read_kb":62113,"num_write":514,"num_write_kb":29400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.14","version":"136'577","reported_seq":8414,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.821844+0000","last_change":"2024-09-15T00:11:16.822160+0000","last_active":"2024-09-15T00:11:16.821844+0000","last_peered":"2024-09-15T00:11:16.821844+0000","last_clean":"2024-09-15T00:11:16.821844+0000","last_became_active":"2024-09-15T00:10:58.353801+0000","last_became_peered":"2024-09-15T00:10:58.353801+0000","last_unstale":"2024-09-15T00:11:16.821844+0000","last_undegraded":"2024-09-15T00:11:16.821844+0000","last_fullsized":"2024-09-15T00:11:16.821844+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:56:52.432800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503938,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9435,"num_read_kb":45249,"num_write":397,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":266,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.821165+0000","last_change":"2024-09-15T00:10:58.352604+0000","last_active":"2024-09-15T00:11:16.821165+0000","last_peered":"2024-09-15T00:11:16.821165+0000","last_clean":"2024-09-15T00:11:16.821165+0000","last_became_active":"2024-09-15T00:10:58.352314+0000","last_became_peered":"2024-09-15T00:10:58.352314+0000","last_unstale":"2024-09-15T00:11:16.821165+0000","last_undegraded":"2024-09-15T00:11:16.821165+0000","last_fullsized":"2024-09-15T00:11:16.821165+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:11.117426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"138'455","reported_seq":3520,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.900822+0000","last_change":"2024-09-15T00:11:16.810432+0000","last_active":"2024-09-15T00:11:16.900822+0000","last_peered":"2024-09-15T00:11:16.900822+0000","last_clean":"2024-09-15T00:11:16.900822+0000","last_became_active":"2024-09-15T00:10:58.347739+0000","last_became_peered":"2024-09-15T00:10:58.347739+0000","last_unstale":"2024-09-15T00:11:16.900822+0000","last_undegraded":"2024-09-15T00:11:16.900822+0000","last_fullsized":"2024-09-15T00:11:16.900822+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:22:34.686812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099603400000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2834,"num_read_kb":21154,"num_write":404,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":259,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.809276+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:11:16.809276+0000","last_peered":"2024-09-15T00:11:16.809276+0000","last_clean":"2024-09-15T00:11:16.809276+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:11:16.809276+0000","last_undegraded":"2024-09-15T00:11:16.809276+0000","last_fullsized":"2024-09-15T00:11:16.809276+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'301","reported_seq":3560,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825333+0000","last_change":"2024-09-15T00:11:16.808663+0000","last_active":"2024-09-15T00:11:17.825333+0000","last_peered":"2024-09-15T00:11:17.825333+0000","last_clean":"2024-09-15T00:11:17.825333+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:11:17.825333+0000","last_undegraded":"2024-09-15T00:11:17.825333+0000","last_fullsized":"2024-09-15T00:11:17.825333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:52.093302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3137,"num_read_kb":36813,"num_write":322,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":273,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820766+0000","last_change":"2024-09-15T00:10:58.352137+0000","last_active":"2024-09-15T00:11:16.820766+0000","last_peered":"2024-09-15T00:11:16.820766+0000","last_clean":"2024-09-15T00:11:16.820766+0000","last_became_active":"2024-09-15T00:10:58.351850+0000","last_became_peered":"2024-09-15T00:10:58.351850+0000","last_unstale":"2024-09-15T00:11:16.820766+0000","last_undegraded":"2024-09-15T00:11:16.820766+0000","last_fullsized":"2024-09-15T00:11:16.820766+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:24:33.533264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"138'804","reported_seq":7224,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.911446+0000","last_change":"2024-09-15T00:11:16.810988+0000","last_active":"2024-09-15T00:11:16.911446+0000","last_peered":"2024-09-15T00:11:16.911446+0000","last_clean":"2024-09-15T00:11:16.911446+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:11:16.911446+0000","last_undegraded":"2024-09-15T00:11:16.911446+0000","last_fullsized":"2024-09-15T00:11:16.911446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:19:00.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015707939999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9943,"num_read_kb":67757,"num_write":840,"num_write_kb":50219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":273,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.809509+0000","last_change":"2024-09-15T00:10:58.347517+0000","last_active":"2024-09-15T00:11:16.809509+0000","last_peered":"2024-09-15T00:11:16.809509+0000","last_clean":"2024-09-15T00:11:16.809509+0000","last_became_active":"2024-09-15T00:10:58.347211+0000","last_became_peered":"2024-09-15T00:10:58.347211+0000","last_unstale":"2024-09-15T00:11:16.809509+0000","last_undegraded":"2024-09-15T00:11:16.809509+0000","last_fullsized":"2024-09-15T00:11:16.809509+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:26:43.176312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'598","reported_seq":5944,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825730+0000","last_change":"2024-09-15T00:11:16.819731+0000","last_active":"2024-09-15T00:11:17.825730+0000","last_peered":"2024-09-15T00:11:17.825730+0000","last_clean":"2024-09-15T00:11:17.825730+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:11:17.825730+0000","last_undegraded":"2024-09-15T00:11:17.825730+0000","last_fullsized":"2024-09-15T00:11:17.825730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:56.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022079199999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6563,"num_read_kb":34828,"num_write":657,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.3","version":"137'665","reported_seq":5844,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.931451+0000","last_change":"2024-09-15T00:11:15.822499+0000","last_active":"2024-09-15T00:11:15.931451+0000","last_peered":"2024-09-15T00:11:15.931451+0000","last_clean":"2024-09-15T00:11:15.931451+0000","last_became_active":"2024-09-15T00:11:01.373244+0000","last_became_peered":"2024-09-15T00:11:01.373244+0000","last_unstale":"2024-09-15T00:11:15.931451+0000","last_undegraded":"2024-09-15T00:11:15.931451+0000","last_fullsized":"2024-09-15T00:11:15.931451+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:37:41.407683+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003194878,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7226,"num_read_kb":26558,"num_write":561,"num_write_kb":3418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"2.2","version":"18'2","reported_seq":264,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825612+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:11:17.825612+0000","last_peered":"2024-09-15T00:11:17.825612+0000","last_clean":"2024-09-15T00:11:17.825612+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:11:17.825612+0000","last_undegraded":"2024-09-15T00:11:17.825612+0000","last_fullsized":"2024-09-15T00:11:17.825612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"136'714","reported_seq":7116,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.811053+0000","last_change":"2024-09-15T00:11:16.811546+0000","last_active":"2024-09-15T00:11:16.811053+0000","last_peered":"2024-09-15T00:11:16.811053+0000","last_clean":"2024-09-15T00:11:16.811053+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:11:16.811053+0000","last_undegraded":"2024-09-15T00:11:16.811053+0000","last_fullsized":"2024-09-15T00:11:16.811053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:46.467190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570948,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7803,"num_read_kb":54109,"num_write":924,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":262,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825100+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:11:17.825100+0000","last_peered":"2024-09-15T00:11:17.825100+0000","last_clean":"2024-09-15T00:11:17.825100+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:11:17.825100+0000","last_undegraded":"2024-09-15T00:11:17.825100+0000","last_fullsized":"2024-09-15T00:11:17.825100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'627","reported_seq":3685,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.832500+0000","last_change":"2024-09-15T00:11:15.833663+0000","last_active":"2024-09-15T00:11:15.832500+0000","last_peered":"2024-09-15T00:11:15.832500+0000","last_clean":"2024-09-15T00:11:15.832500+0000","last_became_active":"2024-09-15T00:10:58.346965+0000","last_became_peered":"2024-09-15T00:10:58.346965+0000","last_unstale":"2024-09-15T00:11:15.832500+0000","last_undegraded":"2024-09-15T00:11:15.832500+0000","last_fullsized":"2024-09-15T00:11:15.832500+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:13:36.312618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014410243,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5290,"num_read_kb":27132,"num_write":878,"num_write_kb":8037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":2365199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":259,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820385+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:11:16.820385+0000","last_peered":"2024-09-15T00:11:16.820385+0000","last_clean":"2024-09-15T00:11:16.820385+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:11:16.820385+0000","last_undegraded":"2024-09-15T00:11:16.820385+0000","last_fullsized":"2024-09-15T00:11:16.820385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'493","reported_seq":6837,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.810490+0000","last_change":"2024-09-15T00:11:16.810764+0000","last_active":"2024-09-15T00:11:16.810490+0000","last_peered":"2024-09-15T00:11:16.810490+0000","last_clean":"2024-09-15T00:11:16.810490+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:11:16.810490+0000","last_undegraded":"2024-09-15T00:11:16.810490+0000","last_fullsized":"2024-09-15T00:11:16.810490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:10:05.221091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215973,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6232,"num_read_kb":25476,"num_write":407,"num_write_kb":5284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.3","version":"16'1","reported_seq":263,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825629+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:11:17.825629+0000","last_peered":"2024-09-15T00:11:17.825629+0000","last_clean":"2024-09-15T00:11:17.825629+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:11:17.825629+0000","last_undegraded":"2024-09-15T00:11:17.825629+0000","last_fullsized":"2024-09-15T00:11:17.825629+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":425,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820338+0000","last_change":"2024-09-15T00:10:26.160336+0000","last_active":"2024-09-15T00:11:16.820338+0000","last_peered":"2024-09-15T00:11:16.820338+0000","last_clean":"2024-09-15T00:11:16.820338+0000","last_became_active":"2024-09-15T00:10:26.159969+0000","last_became_peered":"2024-09-15T00:10:26.159969+0000","last_unstale":"2024-09-15T00:11:16.820338+0000","last_undegraded":"2024-09-15T00:11:16.820338+0000","last_fullsized":"2024-09-15T00:11:16.820338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:40:41.381426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"136'570","reported_seq":11072,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825039+0000","last_change":"2024-09-15T00:11:16.808250+0000","last_active":"2024-09-15T00:11:17.825039+0000","last_peered":"2024-09-15T00:11:17.825039+0000","last_clean":"2024-09-15T00:11:17.825039+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:11:17.825039+0000","last_undegraded":"2024-09-15T00:11:17.825039+0000","last_fullsized":"2024-09-15T00:11:17.825039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:18.362309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098862600000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":12740,"num_read_kb":88117,"num_write":607,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.9","version":"137'511","reported_seq":8981,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.914013+0000","last_change":"2024-09-15T00:11:15.832454+0000","last_active":"2024-09-15T00:11:15.914013+0000","last_peered":"2024-09-15T00:11:15.914013+0000","last_clean":"2024-09-15T00:11:15.914013+0000","last_became_active":"2024-09-15T00:10:58.346994+0000","last_became_peered":"2024-09-15T00:10:58.346994+0000","last_unstale":"2024-09-15T00:11:15.914013+0000","last_undegraded":"2024-09-15T00:11:15.914013+0000","last_fullsized":"2024-09-15T00:11:15.914013+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:23:58.491297+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013263183,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9287,"num_read_kb":49680,"num_write":419,"num_write_kb":13143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"3.a","version":"137'346","reported_seq":2490,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.822249+0000","last_change":"2024-09-15T00:11:16.822630+0000","last_active":"2024-09-15T00:11:16.822249+0000","last_peered":"2024-09-15T00:11:16.822249+0000","last_clean":"2024-09-15T00:11:16.822249+0000","last_became_active":"2024-09-15T00:10:58.353160+0000","last_became_peered":"2024-09-15T00:10:58.353160+0000","last_unstale":"2024-09-15T00:11:16.822249+0000","last_undegraded":"2024-09-15T00:11:16.822249+0000","last_fullsized":"2024-09-15T00:11:16.822249+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:32.587056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001573252,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2498,"num_read_kb":16284,"num_write":316,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.b","version":"137'342","reported_seq":3844,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.810051+0000","last_change":"2024-09-15T00:11:16.810051+0000","last_active":"2024-09-15T00:11:16.810051+0000","last_peered":"2024-09-15T00:11:16.810051+0000","last_clean":"2024-09-15T00:11:16.810051+0000","last_became_active":"2024-09-15T00:10:58.347346+0000","last_became_peered":"2024-09-15T00:10:58.347346+0000","last_unstale":"2024-09-15T00:11:16.810051+0000","last_undegraded":"2024-09-15T00:11:16.810051+0000","last_fullsized":"2024-09-15T00:11:16.810051+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:18:30.185709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000721333,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3470,"num_read_kb":33354,"num_write":301,"num_write_kb":6941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.c","version":"138'1089","reported_seq":5910,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825846+0000","last_change":"2024-09-15T00:11:16.819660+0000","last_active":"2024-09-15T00:11:17.825846+0000","last_peered":"2024-09-15T00:11:17.825846+0000","last_clean":"2024-09-15T00:11:17.825846+0000","last_became_active":"2024-09-15T00:10:26.159829+0000","last_became_peered":"2024-09-15T00:10:26.159829+0000","last_unstale":"2024-09-15T00:11:17.825846+0000","last_undegraded":"2024-09-15T00:11:17.825846+0000","last_fullsized":"2024-09-15T00:11:17.825846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1089,"log_dups_size":0,"ondisk_log_size":1089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:32:02.389557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021428929999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8751,"num_read_kb":31066,"num_write":1650,"num_write_kb":11591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.d","version":"137'472","reported_seq":5140,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.821809+0000","last_change":"2024-09-15T00:11:16.821809+0000","last_active":"2024-09-15T00:11:16.821809+0000","last_peered":"2024-09-15T00:11:16.821809+0000","last_clean":"2024-09-15T00:11:16.821809+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:11:16.821809+0000","last_undegraded":"2024-09-15T00:11:16.821809+0000","last_fullsized":"2024-09-15T00:11:16.821809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:55:03.400561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012186440000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6281,"num_read_kb":25421,"num_write":438,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.e","version":"137'321","reported_seq":4451,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825800+0000","last_change":"2024-09-15T00:11:16.818687+0000","last_active":"2024-09-15T00:11:17.825800+0000","last_peered":"2024-09-15T00:11:17.825800+0000","last_clean":"2024-09-15T00:11:17.825800+0000","last_became_active":"2024-09-15T00:10:58.347480+0000","last_became_peered":"2024-09-15T00:10:58.347480+0000","last_unstale":"2024-09-15T00:11:17.825800+0000","last_undegraded":"2024-09-15T00:11:17.825800+0000","last_fullsized":"2024-09-15T00:11:17.825800+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:57.538275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015201839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3806,"num_read_kb":22330,"num_write":293,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.f","version":"135'777","reported_seq":5180,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825867+0000","last_change":"2024-09-15T00:11:16.819308+0000","last_active":"2024-09-15T00:11:17.825867+0000","last_peered":"2024-09-15T00:11:17.825867+0000","last_clean":"2024-09-15T00:11:17.825867+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:11:17.825867+0000","last_undegraded":"2024-09-15T00:11:17.825867+0000","last_fullsized":"2024-09-15T00:11:17.825867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:38.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018611719999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7063,"num_read_kb":22283,"num_write":819,"num_write_kb":6499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.10","version":"136'345","reported_seq":5995,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825594+0000","last_change":"2024-09-15T00:11:16.818063+0000","last_active":"2024-09-15T00:11:17.825594+0000","last_peered":"2024-09-15T00:11:17.825594+0000","last_clean":"2024-09-15T00:11:17.825594+0000","last_became_active":"2024-09-15T00:10:26.161430+0000","last_became_peered":"2024-09-15T00:10:26.161430+0000","last_unstale":"2024-09-15T00:11:17.825594+0000","last_undegraded":"2024-09-15T00:11:17.825594+0000","last_fullsized":"2024-09-15T00:11:17.825594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:05:05.765250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014647,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5082,"num_read_kb":32663,"num_write":383,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.11","version":"134'1204","reported_seq":4787,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.821481+0000","last_change":"2024-09-15T00:11:16.821784+0000","last_active":"2024-09-15T00:11:16.821481+0000","last_peered":"2024-09-15T00:11:16.821481+0000","last_clean":"2024-09-15T00:11:16.821481+0000","last_became_active":"2024-09-15T00:10:58.354658+0000","last_became_peered":"2024-09-15T00:10:58.354658+0000","last_unstale":"2024-09-15T00:11:16.821481+0000","last_undegraded":"2024-09-15T00:11:16.821481+0000","last_fullsized":"2024-09-15T00:11:16.821481+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:36.660704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185029,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7974,"num_read_kb":21277,"num_write":1815,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.12","version":"136'1278","reported_seq":10300,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820949+0000","last_change":"2024-09-15T00:11:16.821367+0000","last_active":"2024-09-15T00:11:16.820949+0000","last_peered":"2024-09-15T00:11:16.820949+0000","last_clean":"2024-09-15T00:11:16.820949+0000","last_became_active":"2024-09-15T00:10:26.160011+0000","last_became_peered":"2024-09-15T00:10:26.160011+0000","last_unstale":"2024-09-15T00:11:16.820949+0000","last_undegraded":"2024-09-15T00:11:16.820949+0000","last_fullsized":"2024-09-15T00:11:16.820949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:11.992628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079433800000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":12900,"num_read_kb":112467,"num_write":1928,"num_write_kb":43546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.13","version":"137'243","reported_seq":2461,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.822364+0000","last_change":"2024-09-15T00:11:16.822816+0000","last_active":"2024-09-15T00:11:16.822364+0000","last_peered":"2024-09-15T00:11:16.822364+0000","last_clean":"2024-09-15T00:11:16.822364+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:11:16.822364+0000","last_undegraded":"2024-09-15T00:11:16.822364+0000","last_fullsized":"2024-09-15T00:11:16.822364+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:36:47.914784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013592509999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1867,"num_read_kb":15504,"num_write":229,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":79,"num_read":254880,"num_read_kb":1398134,"num_write":21734,"num_write_kb":428106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1261,"num_bytes_recovered":92274536,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9297920,"data_stored":90672039,"data_compressed":2316230,"data_compressed_allocated":6270976,"data_compressed_original":87500800,"omap_allocated":11040,"internal_metadata":0},"log_size":19669,"ondisk_log_size":19669,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":26618,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640290,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59192,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312648,"statfs":{"total":96636764160,"available":96576151552,"internally_reserved":0,"allocated":1650688,"data_stored":693156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":118,"seq":506806140933,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":1680,"kb_used_omap":31,"kb_used_meta":32352,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":1720320,"data_stored":958761,"data_compressed":1208,"data_compressed_allocated":28672,"data_compressed_original":237568,"omap_allocated":31754,"internal_metadata":33129462},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77272,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294568,"statfs":{"total":96636764160,"available":96557637632,"internally_reserved":0,"allocated":1703936,"data_stored":1283505,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640290,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1646592,"data_stored":693137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37008,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334832,"statfs":{"total":96636764160,"available":96598867968,"internally_reserved":0,"allocated":1695744,"data_stored":1283543,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":571230650370,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9273344,"data_stored":90570113,"data_compressed":2315776,"data_compressed_allocated":6262784,"data_compressed_original":87402496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":101907,"data_compressed":454,"data_compressed_allocated":8192,"data_compressed_original":98304,"omap_allocated":11040,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:21.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:21.522 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:21.522 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:11:21.533 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-15T00:11:20.878938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":79,"num_read":254996,"num_read_kb":1398234,"num_write":21895,"num_write_kb":430524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1263,"num_bytes_recovered":92864904,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19789,"ondisk_log_size":19789,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":266728,"kb_used_data":8220,"kb_used_omap":37,"kb_used_meta":258330,"kb_avail":471592472,"statfs":{"total":483183820800,"available":482910691328,"internally_reserved":0,"allocated":8417280,"data_stored":4912102,"data_compressed":19298,"data_compressed_allocated":102400,"data_compressed_original":1400832,"omap_allocated":38132,"internal_metadata":264530700},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-41133580,"num_objects":-143,"num_object_clones":-41,"num_object_copies":-286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-143,"num_whiteouts":-35,"num_read":853,"num_read_kb":723,"num_write":185,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000884"},"pg_stats":[{"pgid":"3.1f","version":"138'535","reported_seq":11553,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.822559+0000","last_change":"2024-09-15T00:11:16.822559+0000","last_active":"2024-09-15T00:11:16.822559+0000","last_peered":"2024-09-15T00:11:16.822559+0000","last_clean":"2024-09-15T00:11:16.822559+0000","last_became_active":"2024-09-15T00:10:26.157785+0000","last_became_peered":"2024-09-15T00:10:26.157785+0000","last_unstale":"2024-09-15T00:11:16.822559+0000","last_undegraded":"2024-09-15T00:11:16.822559+0000","last_fullsized":"2024-09-15T00:11:16.822559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:45:18.226279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001206129,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12081,"num_read_kb":64593,"num_write":535,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1e","version":"135'496","reported_seq":5487,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825201+0000","last_change":"2024-09-15T00:11:16.808902+0000","last_active":"2024-09-15T00:11:17.825201+0000","last_peered":"2024-09-15T00:11:17.825201+0000","last_clean":"2024-09-15T00:11:17.825201+0000","last_became_active":"2024-09-15T00:10:58.347628+0000","last_became_peered":"2024-09-15T00:10:58.347628+0000","last_unstale":"2024-09-15T00:11:17.825201+0000","last_undegraded":"2024-09-15T00:11:17.825201+0000","last_fullsized":"2024-09-15T00:11:17.825201+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:19.680950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014193910000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6067,"num_read_kb":40333,"num_write":376,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1d","version":"137'826","reported_seq":5668,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825771+0000","last_change":"2024-09-15T00:11:16.819173+0000","last_active":"2024-09-15T00:11:17.825771+0000","last_peered":"2024-09-15T00:11:17.825771+0000","last_clean":"2024-09-15T00:11:17.825771+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:11:17.825771+0000","last_undegraded":"2024-09-15T00:11:17.825771+0000","last_fullsized":"2024-09-15T00:11:17.825771+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:38.089567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019789569999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8748,"num_read_kb":37428,"num_write":973,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1c","version":"138'512","reported_seq":6088,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825601+0000","last_change":"2024-09-15T00:11:16.818860+0000","last_active":"2024-09-15T00:11:17.825601+0000","last_peered":"2024-09-15T00:11:17.825601+0000","last_clean":"2024-09-15T00:11:17.825601+0000","last_became_active":"2024-09-15T00:10:58.347433+0000","last_became_peered":"2024-09-15T00:10:58.347433+0000","last_unstale":"2024-09-15T00:11:17.825601+0000","last_undegraded":"2024-09-15T00:11:17.825601+0000","last_fullsized":"2024-09-15T00:11:17.825601+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:34.872358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878240000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6371,"num_read_kb":39648,"num_write":614,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1b","version":"137'519","reported_seq":10455,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.835389+0000","last_change":"2024-09-15T00:11:15.835389+0000","last_active":"2024-09-15T00:11:15.835389+0000","last_peered":"2024-09-15T00:11:15.835389+0000","last_clean":"2024-09-15T00:11:15.835389+0000","last_became_active":"2024-09-15T00:10:58.352065+0000","last_became_peered":"2024-09-15T00:10:58.352065+0000","last_unstale":"2024-09-15T00:11:15.835389+0000","last_undegraded":"2024-09-15T00:11:15.835389+0000","last_fullsized":"2024-09-15T00:11:15.835389+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:06:28.305710+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016246954000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11088,"num_read_kb":77879,"num_write":574,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"3.1a","version":"137'534","reported_seq":11068,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.961475+0000","last_change":"2024-09-15T00:11:15.826478+0000","last_active":"2024-09-15T00:11:15.961475+0000","last_peered":"2024-09-15T00:11:15.961475+0000","last_clean":"2024-09-15T00:11:15.961475+0000","last_became_active":"2024-09-15T00:10:58.346770+0000","last_became_peered":"2024-09-15T00:10:58.346770+0000","last_unstale":"2024-09-15T00:11:15.961475+0000","last_undegraded":"2024-09-15T00:11:15.961475+0000","last_fullsized":"2024-09-15T00:11:15.961475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:03.547839+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0073748930000000004,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":13906,"num_read_kb":40584,"num_write":444,"num_write_kb":8754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"3.19","version":"136'709","reported_seq":10753,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.810826+0000","last_change":"2024-09-15T00:11:16.811299+0000","last_active":"2024-09-15T00:11:16.810826+0000","last_peered":"2024-09-15T00:11:16.810826+0000","last_clean":"2024-09-15T00:11:16.810826+0000","last_became_active":"2024-09-15T00:10:58.350007+0000","last_became_peered":"2024-09-15T00:10:58.350007+0000","last_unstale":"2024-09-15T00:11:16.810826+0000","last_undegraded":"2024-09-15T00:11:16.810826+0000","last_fullsized":"2024-09-15T00:11:16.810826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:11.344098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019477310000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13597,"num_read_kb":55399,"num_write":535,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.18","version":"137'655","reported_seq":8623,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.824975+0000","last_change":"2024-09-15T00:11:16.808297+0000","last_active":"2024-09-15T00:11:17.824975+0000","last_peered":"2024-09-15T00:11:17.824975+0000","last_clean":"2024-09-15T00:11:17.824975+0000","last_became_active":"2024-09-15T00:10:26.158999+0000","last_became_peered":"2024-09-15T00:10:26.158999+0000","last_unstale":"2024-09-15T00:11:17.824975+0000","last_undegraded":"2024-09-15T00:11:17.824975+0000","last_fullsized":"2024-09-15T00:11:17.824975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:11:07.970440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012280349999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9415,"num_read_kb":64591,"num_write":699,"num_write_kb":14239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.17","version":"136'859","reported_seq":15029,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825763+0000","last_change":"2024-09-15T00:11:16.818448+0000","last_active":"2024-09-15T00:11:17.825763+0000","last_peered":"2024-09-15T00:11:17.825763+0000","last_clean":"2024-09-15T00:11:17.825763+0000","last_became_active":"2024-09-15T00:10:26.158796+0000","last_became_peered":"2024-09-15T00:10:26.158796+0000","last_unstale":"2024-09-15T00:11:17.825763+0000","last_undegraded":"2024-09-15T00:11:17.825763+0000","last_fullsized":"2024-09-15T00:11:17.825763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:27.044437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001285972,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":16900,"num_read_kb":72084,"num_write":902,"num_write_kb":44641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.16","version":"135'832","reported_seq":8136,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825736+0000","last_change":"2024-09-15T00:11:16.818288+0000","last_active":"2024-09-15T00:11:17.825736+0000","last_peered":"2024-09-15T00:11:17.825736+0000","last_clean":"2024-09-15T00:11:17.825736+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:11:17.825736+0000","last_undegraded":"2024-09-15T00:11:17.825736+0000","last_fullsized":"2024-09-15T00:11:17.825736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.421381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098720300000000008,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10216,"num_read_kb":33960,"num_write":980,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.15","version":"136'460","reported_seq":6838,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820887+0000","last_change":"2024-09-15T00:11:16.821358+0000","last_active":"2024-09-15T00:11:16.820887+0000","last_peered":"2024-09-15T00:11:16.820887+0000","last_clean":"2024-09-15T00:11:16.820887+0000","last_became_active":"2024-09-15T00:10:58.351863+0000","last_became_peered":"2024-09-15T00:10:58.351863+0000","last_unstale":"2024-09-15T00:11:16.820887+0000","last_undegraded":"2024-09-15T00:11:16.820887+0000","last_fullsized":"2024-09-15T00:11:16.820887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:36:31.765520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081705799999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6309,"num_read_kb":62113,"num_write":514,"num_write_kb":29400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.14","version":"136'577","reported_seq":8414,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.821844+0000","last_change":"2024-09-15T00:11:16.822160+0000","last_active":"2024-09-15T00:11:16.821844+0000","last_peered":"2024-09-15T00:11:16.821844+0000","last_clean":"2024-09-15T00:11:16.821844+0000","last_became_active":"2024-09-15T00:10:58.353801+0000","last_became_peered":"2024-09-15T00:10:58.353801+0000","last_unstale":"2024-09-15T00:11:16.821844+0000","last_undegraded":"2024-09-15T00:11:16.821844+0000","last_fullsized":"2024-09-15T00:11:16.821844+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:56:52.432800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503938,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9435,"num_read_kb":45249,"num_write":397,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":266,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.821165+0000","last_change":"2024-09-15T00:10:58.352604+0000","last_active":"2024-09-15T00:11:16.821165+0000","last_peered":"2024-09-15T00:11:16.821165+0000","last_clean":"2024-09-15T00:11:16.821165+0000","last_became_active":"2024-09-15T00:10:58.352314+0000","last_became_peered":"2024-09-15T00:10:58.352314+0000","last_unstale":"2024-09-15T00:11:16.821165+0000","last_undegraded":"2024-09-15T00:11:16.821165+0000","last_fullsized":"2024-09-15T00:11:16.821165+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:11.117426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"138'455","reported_seq":3520,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.900822+0000","last_change":"2024-09-15T00:11:16.810432+0000","last_active":"2024-09-15T00:11:16.900822+0000","last_peered":"2024-09-15T00:11:16.900822+0000","last_clean":"2024-09-15T00:11:16.900822+0000","last_became_active":"2024-09-15T00:10:58.347739+0000","last_became_peered":"2024-09-15T00:10:58.347739+0000","last_unstale":"2024-09-15T00:11:16.900822+0000","last_undegraded":"2024-09-15T00:11:16.900822+0000","last_fullsized":"2024-09-15T00:11:16.900822+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:22:34.686812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099603400000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2834,"num_read_kb":21154,"num_write":404,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":259,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.809276+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:11:16.809276+0000","last_peered":"2024-09-15T00:11:16.809276+0000","last_clean":"2024-09-15T00:11:16.809276+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:11:16.809276+0000","last_undegraded":"2024-09-15T00:11:16.809276+0000","last_fullsized":"2024-09-15T00:11:16.809276+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'301","reported_seq":3560,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825333+0000","last_change":"2024-09-15T00:11:16.808663+0000","last_active":"2024-09-15T00:11:17.825333+0000","last_peered":"2024-09-15T00:11:17.825333+0000","last_clean":"2024-09-15T00:11:17.825333+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:11:17.825333+0000","last_undegraded":"2024-09-15T00:11:17.825333+0000","last_fullsized":"2024-09-15T00:11:17.825333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:52.093302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3137,"num_read_kb":36813,"num_write":322,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":273,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820766+0000","last_change":"2024-09-15T00:10:58.352137+0000","last_active":"2024-09-15T00:11:16.820766+0000","last_peered":"2024-09-15T00:11:16.820766+0000","last_clean":"2024-09-15T00:11:16.820766+0000","last_became_active":"2024-09-15T00:10:58.351850+0000","last_became_peered":"2024-09-15T00:10:58.351850+0000","last_unstale":"2024-09-15T00:11:16.820766+0000","last_undegraded":"2024-09-15T00:11:16.820766+0000","last_fullsized":"2024-09-15T00:11:16.820766+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:24:33.533264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"138'804","reported_seq":7224,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.911446+0000","last_change":"2024-09-15T00:11:16.810988+0000","last_active":"2024-09-15T00:11:16.911446+0000","last_peered":"2024-09-15T00:11:16.911446+0000","last_clean":"2024-09-15T00:11:16.911446+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:11:16.911446+0000","last_undegraded":"2024-09-15T00:11:16.911446+0000","last_fullsized":"2024-09-15T00:11:16.911446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:19:00.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015707939999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9943,"num_read_kb":67757,"num_write":840,"num_write_kb":50219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":273,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.809509+0000","last_change":"2024-09-15T00:10:58.347517+0000","last_active":"2024-09-15T00:11:16.809509+0000","last_peered":"2024-09-15T00:11:16.809509+0000","last_clean":"2024-09-15T00:11:16.809509+0000","last_became_active":"2024-09-15T00:10:58.347211+0000","last_became_peered":"2024-09-15T00:10:58.347211+0000","last_unstale":"2024-09-15T00:11:16.809509+0000","last_undegraded":"2024-09-15T00:11:16.809509+0000","last_fullsized":"2024-09-15T00:11:16.809509+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:26:43.176312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'598","reported_seq":5944,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825730+0000","last_change":"2024-09-15T00:11:16.819731+0000","last_active":"2024-09-15T00:11:17.825730+0000","last_peered":"2024-09-15T00:11:17.825730+0000","last_clean":"2024-09-15T00:11:17.825730+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:11:17.825730+0000","last_undegraded":"2024-09-15T00:11:17.825730+0000","last_fullsized":"2024-09-15T00:11:17.825730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:56.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022079199999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6563,"num_read_kb":34828,"num_write":657,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.3","version":"137'665","reported_seq":5844,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.931451+0000","last_change":"2024-09-15T00:11:15.822499+0000","last_active":"2024-09-15T00:11:15.931451+0000","last_peered":"2024-09-15T00:11:15.931451+0000","last_clean":"2024-09-15T00:11:15.931451+0000","last_became_active":"2024-09-15T00:11:01.373244+0000","last_became_peered":"2024-09-15T00:11:01.373244+0000","last_unstale":"2024-09-15T00:11:15.931451+0000","last_undegraded":"2024-09-15T00:11:15.931451+0000","last_fullsized":"2024-09-15T00:11:15.931451+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:37:41.407683+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003194878,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7226,"num_read_kb":26558,"num_write":561,"num_write_kb":3418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"2.2","version":"18'2","reported_seq":264,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825612+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:11:17.825612+0000","last_peered":"2024-09-15T00:11:17.825612+0000","last_clean":"2024-09-15T00:11:17.825612+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:11:17.825612+0000","last_undegraded":"2024-09-15T00:11:17.825612+0000","last_fullsized":"2024-09-15T00:11:17.825612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"136'714","reported_seq":7116,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.811053+0000","last_change":"2024-09-15T00:11:16.811546+0000","last_active":"2024-09-15T00:11:16.811053+0000","last_peered":"2024-09-15T00:11:16.811053+0000","last_clean":"2024-09-15T00:11:16.811053+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:11:16.811053+0000","last_undegraded":"2024-09-15T00:11:16.811053+0000","last_fullsized":"2024-09-15T00:11:16.811053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:46.467190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570948,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7803,"num_read_kb":54109,"num_write":924,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":262,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825100+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:11:17.825100+0000","last_peered":"2024-09-15T00:11:17.825100+0000","last_clean":"2024-09-15T00:11:17.825100+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:11:17.825100+0000","last_undegraded":"2024-09-15T00:11:17.825100+0000","last_fullsized":"2024-09-15T00:11:17.825100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'627","reported_seq":3685,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.832500+0000","last_change":"2024-09-15T00:11:15.833663+0000","last_active":"2024-09-15T00:11:15.832500+0000","last_peered":"2024-09-15T00:11:15.832500+0000","last_clean":"2024-09-15T00:11:15.832500+0000","last_became_active":"2024-09-15T00:10:58.346965+0000","last_became_peered":"2024-09-15T00:10:58.346965+0000","last_unstale":"2024-09-15T00:11:15.832500+0000","last_undegraded":"2024-09-15T00:11:15.832500+0000","last_fullsized":"2024-09-15T00:11:15.832500+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:13:36.312618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014410243,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5290,"num_read_kb":27132,"num_write":878,"num_write_kb":8037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":2365199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":259,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820385+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:11:16.820385+0000","last_peered":"2024-09-15T00:11:16.820385+0000","last_clean":"2024-09-15T00:11:16.820385+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:11:16.820385+0000","last_undegraded":"2024-09-15T00:11:16.820385+0000","last_fullsized":"2024-09-15T00:11:16.820385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'493","reported_seq":6837,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.810490+0000","last_change":"2024-09-15T00:11:16.810764+0000","last_active":"2024-09-15T00:11:16.810490+0000","last_peered":"2024-09-15T00:11:16.810490+0000","last_clean":"2024-09-15T00:11:16.810490+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:11:16.810490+0000","last_undegraded":"2024-09-15T00:11:16.810490+0000","last_fullsized":"2024-09-15T00:11:16.810490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:10:05.221091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215973,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6232,"num_read_kb":25476,"num_write":407,"num_write_kb":5284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.3","version":"16'1","reported_seq":263,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825629+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:11:17.825629+0000","last_peered":"2024-09-15T00:11:17.825629+0000","last_clean":"2024-09-15T00:11:17.825629+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:11:17.825629+0000","last_undegraded":"2024-09-15T00:11:17.825629+0000","last_fullsized":"2024-09-15T00:11:17.825629+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":425,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820338+0000","last_change":"2024-09-15T00:10:26.160336+0000","last_active":"2024-09-15T00:11:16.820338+0000","last_peered":"2024-09-15T00:11:16.820338+0000","last_clean":"2024-09-15T00:11:16.820338+0000","last_became_active":"2024-09-15T00:10:26.159969+0000","last_became_peered":"2024-09-15T00:10:26.159969+0000","last_unstale":"2024-09-15T00:11:16.820338+0000","last_undegraded":"2024-09-15T00:11:16.820338+0000","last_fullsized":"2024-09-15T00:11:16.820338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:40:41.381426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"136'570","reported_seq":11072,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825039+0000","last_change":"2024-09-15T00:11:16.808250+0000","last_active":"2024-09-15T00:11:17.825039+0000","last_peered":"2024-09-15T00:11:17.825039+0000","last_clean":"2024-09-15T00:11:17.825039+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:11:17.825039+0000","last_undegraded":"2024-09-15T00:11:17.825039+0000","last_fullsized":"2024-09-15T00:11:17.825039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:18.362309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098862600000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":12740,"num_read_kb":88117,"num_write":607,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.9","version":"137'511","reported_seq":8981,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-15T00:11:15.914013+0000","last_change":"2024-09-15T00:11:15.832454+0000","last_active":"2024-09-15T00:11:15.914013+0000","last_peered":"2024-09-15T00:11:15.914013+0000","last_clean":"2024-09-15T00:11:15.914013+0000","last_became_active":"2024-09-15T00:10:58.346994+0000","last_became_peered":"2024-09-15T00:10:58.346994+0000","last_unstale":"2024-09-15T00:11:15.914013+0000","last_undegraded":"2024-09-15T00:11:15.914013+0000","last_fullsized":"2024-09-15T00:11:15.914013+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:23:58.491297+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013263183,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9287,"num_read_kb":49680,"num_write":419,"num_write_kb":13143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"4","length":"b"}]},{"pgid":"3.a","version":"137'346","reported_seq":2490,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.822249+0000","last_change":"2024-09-15T00:11:16.822630+0000","last_active":"2024-09-15T00:11:16.822249+0000","last_peered":"2024-09-15T00:11:16.822249+0000","last_clean":"2024-09-15T00:11:16.822249+0000","last_became_active":"2024-09-15T00:10:58.353160+0000","last_became_peered":"2024-09-15T00:10:58.353160+0000","last_unstale":"2024-09-15T00:11:16.822249+0000","last_undegraded":"2024-09-15T00:11:16.822249+0000","last_fullsized":"2024-09-15T00:11:16.822249+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:32.587056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001573252,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2498,"num_read_kb":16284,"num_write":316,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.b","version":"137'342","reported_seq":3844,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.810051+0000","last_change":"2024-09-15T00:11:16.810051+0000","last_active":"2024-09-15T00:11:16.810051+0000","last_peered":"2024-09-15T00:11:16.810051+0000","last_clean":"2024-09-15T00:11:16.810051+0000","last_became_active":"2024-09-15T00:10:58.347346+0000","last_became_peered":"2024-09-15T00:10:58.347346+0000","last_unstale":"2024-09-15T00:11:16.810051+0000","last_undegraded":"2024-09-15T00:11:16.810051+0000","last_fullsized":"2024-09-15T00:11:16.810051+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:18:30.185709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000721333,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3470,"num_read_kb":33354,"num_write":301,"num_write_kb":6941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.c","version":"138'1089","reported_seq":5910,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825846+0000","last_change":"2024-09-15T00:11:16.819660+0000","last_active":"2024-09-15T00:11:17.825846+0000","last_peered":"2024-09-15T00:11:17.825846+0000","last_clean":"2024-09-15T00:11:17.825846+0000","last_became_active":"2024-09-15T00:10:26.159829+0000","last_became_peered":"2024-09-15T00:10:26.159829+0000","last_unstale":"2024-09-15T00:11:17.825846+0000","last_undegraded":"2024-09-15T00:11:17.825846+0000","last_fullsized":"2024-09-15T00:11:17.825846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1089,"log_dups_size":0,"ondisk_log_size":1089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:32:02.389557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021428929999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8751,"num_read_kb":31066,"num_write":1650,"num_write_kb":11591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.d","version":"137'472","reported_seq":5140,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.821809+0000","last_change":"2024-09-15T00:11:16.821809+0000","last_active":"2024-09-15T00:11:16.821809+0000","last_peered":"2024-09-15T00:11:16.821809+0000","last_clean":"2024-09-15T00:11:16.821809+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:11:16.821809+0000","last_undegraded":"2024-09-15T00:11:16.821809+0000","last_fullsized":"2024-09-15T00:11:16.821809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:55:03.400561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012186440000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6281,"num_read_kb":25421,"num_write":438,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.e","version":"137'321","reported_seq":4451,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825800+0000","last_change":"2024-09-15T00:11:16.818687+0000","last_active":"2024-09-15T00:11:17.825800+0000","last_peered":"2024-09-15T00:11:17.825800+0000","last_clean":"2024-09-15T00:11:17.825800+0000","last_became_active":"2024-09-15T00:10:58.347480+0000","last_became_peered":"2024-09-15T00:10:58.347480+0000","last_unstale":"2024-09-15T00:11:17.825800+0000","last_undegraded":"2024-09-15T00:11:17.825800+0000","last_fullsized":"2024-09-15T00:11:17.825800+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:57.538275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015201839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3806,"num_read_kb":22330,"num_write":293,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.f","version":"135'777","reported_seq":5180,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825867+0000","last_change":"2024-09-15T00:11:16.819308+0000","last_active":"2024-09-15T00:11:17.825867+0000","last_peered":"2024-09-15T00:11:17.825867+0000","last_clean":"2024-09-15T00:11:17.825867+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:11:17.825867+0000","last_undegraded":"2024-09-15T00:11:17.825867+0000","last_fullsized":"2024-09-15T00:11:17.825867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:38.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018611719999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7063,"num_read_kb":22283,"num_write":819,"num_write_kb":6499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.10","version":"136'345","reported_seq":5995,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T00:11:17.825594+0000","last_change":"2024-09-15T00:11:16.818063+0000","last_active":"2024-09-15T00:11:17.825594+0000","last_peered":"2024-09-15T00:11:17.825594+0000","last_clean":"2024-09-15T00:11:17.825594+0000","last_became_active":"2024-09-15T00:10:26.161430+0000","last_became_peered":"2024-09-15T00:10:26.161430+0000","last_unstale":"2024-09-15T00:11:17.825594+0000","last_undegraded":"2024-09-15T00:11:17.825594+0000","last_fullsized":"2024-09-15T00:11:17.825594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:05:05.765250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014647,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5082,"num_read_kb":32663,"num_write":383,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.11","version":"134'1204","reported_seq":4787,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.821481+0000","last_change":"2024-09-15T00:11:16.821784+0000","last_active":"2024-09-15T00:11:16.821481+0000","last_peered":"2024-09-15T00:11:16.821481+0000","last_clean":"2024-09-15T00:11:16.821481+0000","last_became_active":"2024-09-15T00:10:58.354658+0000","last_became_peered":"2024-09-15T00:10:58.354658+0000","last_unstale":"2024-09-15T00:11:16.821481+0000","last_undegraded":"2024-09-15T00:11:16.821481+0000","last_fullsized":"2024-09-15T00:11:16.821481+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:36.660704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185029,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7974,"num_read_kb":21277,"num_write":1815,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.12","version":"136'1278","reported_seq":10300,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.820949+0000","last_change":"2024-09-15T00:11:16.821367+0000","last_active":"2024-09-15T00:11:16.820949+0000","last_peered":"2024-09-15T00:11:16.820949+0000","last_clean":"2024-09-15T00:11:16.820949+0000","last_became_active":"2024-09-15T00:10:26.160011+0000","last_became_peered":"2024-09-15T00:10:26.160011+0000","last_unstale":"2024-09-15T00:11:16.820949+0000","last_undegraded":"2024-09-15T00:11:16.820949+0000","last_fullsized":"2024-09-15T00:11:16.820949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:11.992628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079433800000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":12900,"num_read_kb":112467,"num_write":1928,"num_write_kb":43546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.13","version":"137'243","reported_seq":2461,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-15T00:11:16.822364+0000","last_change":"2024-09-15T00:11:16.822816+0000","last_active":"2024-09-15T00:11:16.822364+0000","last_peered":"2024-09-15T00:11:16.822364+0000","last_clean":"2024-09-15T00:11:16.822364+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:11:16.822364+0000","last_undegraded":"2024-09-15T00:11:16.822364+0000","last_fullsized":"2024-09-15T00:11:16.822364+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:36:47.914784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013592509999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1867,"num_read_kb":15504,"num_write":229,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":79,"num_read":254880,"num_read_kb":1398134,"num_write":21734,"num_write_kb":428106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1261,"num_bytes_recovered":92274536,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9297920,"data_stored":90672039,"data_compressed":2316230,"data_compressed_allocated":6270976,"data_compressed_original":87500800,"omap_allocated":11040,"internal_metadata":0},"log_size":19669,"ondisk_log_size":19669,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":26618,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640290,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59192,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312648,"statfs":{"total":96636764160,"available":96576151552,"internally_reserved":0,"allocated":1650688,"data_stored":693156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":118,"seq":506806140933,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":1680,"kb_used_omap":31,"kb_used_meta":32352,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":1720320,"data_stored":958761,"data_compressed":1208,"data_compressed_allocated":28672,"data_compressed_original":237568,"omap_allocated":31754,"internal_metadata":33129462},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77272,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294568,"statfs":{"total":96636764160,"available":96557637632,"internally_reserved":0,"allocated":1703936,"data_stored":1283505,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640290,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1646592,"data_stored":693137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37008,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334832,"statfs":{"total":96636764160,"available":96598867968,"internally_reserved":0,"allocated":1695744,"data_stored":1283543,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":571230650370,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9273344,"data_stored":90570113,"data_compressed":2315776,"data_compressed_allocated":6262784,"data_compressed_original":87402496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":101907,"data_compressed":454,"data_compressed_allocated":8192,"data_compressed_original":98304,"omap_allocated":11040,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:21.535 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T00:11:21.535 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T00:11:21.535 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-15T00:11:21.536 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-15T00:11:21.807 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T00:11:21.840 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-15T00:11:21.838+0000 7f73f660c640 -1 mon.a@0(leader).osd e143 definitely_dead 0 2024-09-15T00:11:22.808 INFO:teuthology.orchestra.run.smithi027.stderr:marked in osd.0. 2024-09-15T00:11:22.827 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-15T00:11:27.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-15T00:11:27.951 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a: {} 2024-09-15T00:11:27.951 INFO:teuthology.orchestra.run.smithi027.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:11:27.961 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b: {} 2024-09-15T00:11:27.961 INFO:teuthology.orchestra.run.smithi027.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:11:27.970 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c: {} 2024-09-15T00:11:27.970 INFO:teuthology.orchestra.run.smithi027.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T00:11:27.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T00:11:28.810 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T00:11:28.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-15T00:11:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-15T00:11:29.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T00:11:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T00:11:30.834 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-15T00:11:31.809 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-15T00:11:31.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T00:11:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T00:11:32.850 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-15T00:11:33.841 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-15T00:11:33.858 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T00:11:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T00:11:34.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-15T00:11:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-15T00:11:35.868 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T00:11:36.860 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T00:11:36.878 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-15T00:11:37.861 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-15T00:11:37.879 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T00:11:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T00:11:38.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-15T00:11:39.874 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-15T00:11:39.889 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-15T00:11:39.890 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-15T00:11:39.890 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-15T00:11:39.890 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-15T00:11:39.890 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T00:11:39.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:11:40.183 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:40.184 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":161,"fsid":"4f532d31-28c1-4236-a74e-6392960baf2b","created":"2024-09-15T00:08:32.007191+0000","modified":"2024-09-15T00:11:39.871757+0000","last_up_change":"2024-09-15T00:11:11.798639+0000","last_in_change":"2024-09-15T00:11:22.802027+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:08:42.896076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:08:49.173060+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-15T00:08:58.879058+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":43,"snap_epoch":138,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"56600dc2-b95d-47a7-b2b1-5ddc54c08341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":55,"up_from":133,"up_thru":149,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6817","nonce":1332878779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6819","nonce":1332878779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6823","nonce":1332878779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6821","nonce":1332878779}]},"public_addr":"172.21.15.27:6817/1332878779","cluster_addr":"172.21.15.27:6819/1332878779","heartbeat_back_addr":"172.21.15.27:6823/1332878779","heartbeat_front_addr":"172.21.15.27:6821/1332878779","state":["exists","up"]},{"osd":1,"uuid":"ea8f0155-8472-4e87-89fd-7716d05065d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6809","nonce":3601510193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6811","nonce":3601510193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6815","nonce":3601510193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6813","nonce":3601510193}]},"public_addr":"172.21.15.27:6809/3601510193","cluster_addr":"172.21.15.27:6811/3601510193","heartbeat_back_addr":"172.21.15.27:6815/3601510193","heartbeat_front_addr":"172.21.15.27:6813/3601510193","state":["exists","up"]},{"osd":2,"uuid":"357d2d59-66f0-47b8-87c8-4acdfe62a07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6801","nonce":3222759256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6803","nonce":3222759256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6807","nonce":3222759256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6805","nonce":3222759256}]},"public_addr":"172.21.15.27:6801/3222759256","cluster_addr":"172.21.15.27:6803/3222759256","heartbeat_back_addr":"172.21.15.27:6807/3222759256","heartbeat_front_addr":"172.21.15.27:6805/3222759256","state":["exists","up"]},{"osd":3,"uuid":"22208f56-d8aa-4a56-a42c-5a990962b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6809","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6811","nonce":675428777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6813","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6816","nonce":675428777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6823","nonce":675428777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6821","nonce":675428777}]},"public_addr":"172.21.15.79:6811/675428777","cluster_addr":"172.21.15.79:6816/675428777","heartbeat_back_addr":"172.21.15.79:6823/675428777","heartbeat_front_addr":"172.21.15.79:6821/675428777","state":["exists","up"]},{"osd":4,"uuid":"a974629e-9ef0-40a3-94bd-382e0bfa8bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":88,"up_from":118,"up_thru":147,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6810","nonce":1046239476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6814","nonce":1046239476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6819","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6820","nonce":1046239476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6815","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6817","nonce":1046239476}]},"public_addr":"172.21.15.79:6810/1046239476","cluster_addr":"172.21.15.79:6814/1046239476","heartbeat_back_addr":"172.21.15.79:6820/1046239476","heartbeat_front_addr":"172.21.15.79:6817/1046239476","state":["exists","up"]},{"osd":5,"uuid":"06b73c2d-466a-4c4e-89fa-31d6eeb1af1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6801","nonce":2235671634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6803","nonce":2235671634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6807","nonce":2235671634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6805","nonce":2235671634}]},"public_addr":"172.21.15.79:6801/2235671634","cluster_addr":"172.21.15.79:6803/2235671634","heartbeat_back_addr":"172.21.15.79:6807/2235671634","heartbeat_front_addr":"172.21.15.79:6805/2235671634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.730107+0000","dead_epoch":56},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.365297+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.868500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.788781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.892143+0000","dead_epoch":90},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.990037+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.12","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:11:40.197 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T00:11:40.197 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:40.485 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":161,"fsid":"4f532d31-28c1-4236-a74e-6392960baf2b","created":"2024-09-15T00:08:32.007191+0000","modified":"2024-09-15T00:11:39.871757+0000","last_up_change":"2024-09-15T00:11:11.798639+0000","last_in_change":"2024-09-15T00:11:22.802027+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:08:42.896076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:08:49.173060+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-15T00:08:58.879058+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":43,"snap_epoch":138,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"56600dc2-b95d-47a7-b2b1-5ddc54c08341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":55,"up_from":133,"up_thru":149,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6817","nonce":1332878779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6819","nonce":1332878779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6823","nonce":1332878779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6821","nonce":1332878779}]},"public_addr":"172.21.15.27:6817/1332878779","cluster_addr":"172.21.15.27:6819/1332878779","heartbeat_back_addr":"172.21.15.27:6823/1332878779","heartbeat_front_addr":"172.21.15.27:6821/1332878779","state":["exists","up"]},{"osd":1,"uuid":"ea8f0155-8472-4e87-89fd-7716d05065d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6809","nonce":3601510193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6811","nonce":3601510193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6815","nonce":3601510193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6813","nonce":3601510193}]},"public_addr":"172.21.15.27:6809/3601510193","cluster_addr":"172.21.15.27:6811/3601510193","heartbeat_back_addr":"172.21.15.27:6815/3601510193","heartbeat_front_addr":"172.21.15.27:6813/3601510193","state":["exists","up"]},{"osd":2,"uuid":"357d2d59-66f0-47b8-87c8-4acdfe62a07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6801","nonce":3222759256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6803","nonce":3222759256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6807","nonce":3222759256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6805","nonce":3222759256}]},"public_addr":"172.21.15.27:6801/3222759256","cluster_addr":"172.21.15.27:6803/3222759256","heartbeat_back_addr":"172.21.15.27:6807/3222759256","heartbeat_front_addr":"172.21.15.27:6805/3222759256","state":["exists","up"]},{"osd":3,"uuid":"22208f56-d8aa-4a56-a42c-5a990962b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6809","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6811","nonce":675428777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6813","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6816","nonce":675428777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6823","nonce":675428777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6821","nonce":675428777}]},"public_addr":"172.21.15.79:6811/675428777","cluster_addr":"172.21.15.79:6816/675428777","heartbeat_back_addr":"172.21.15.79:6823/675428777","heartbeat_front_addr":"172.21.15.79:6821/675428777","state":["exists","up"]},{"osd":4,"uuid":"a974629e-9ef0-40a3-94bd-382e0bfa8bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":88,"up_from":118,"up_thru":147,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6810","nonce":1046239476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6814","nonce":1046239476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6819","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6820","nonce":1046239476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6815","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6817","nonce":1046239476}]},"public_addr":"172.21.15.79:6810/1046239476","cluster_addr":"172.21.15.79:6814/1046239476","heartbeat_back_addr":"172.21.15.79:6820/1046239476","heartbeat_front_addr":"172.21.15.79:6817/1046239476","state":["exists","up"]},{"osd":5,"uuid":"06b73c2d-466a-4c4e-89fa-31d6eeb1af1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6801","nonce":2235671634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6803","nonce":2235671634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6807","nonce":2235671634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6805","nonce":2235671634}]},"public_addr":"172.21.15.79:6801/2235671634","cluster_addr":"172.21.15.79:6803/2235671634","heartbeat_back_addr":"172.21.15.79:6807/2235671634","heartbeat_front_addr":"172.21.15.79:6805/2235671634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.730107+0000","dead_epoch":56},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.365297+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.868500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.788781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.892143+0000","dead_epoch":90},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.990037+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.12","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:11:40.498 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T00:11:40.498 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T00:11:40.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T00:11:40.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T00:11:40.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T00:11:40.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T00:11:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:47244640296 2024-09-15T00:11:40.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T00:11:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:47244640295 2024-09-15T00:11:40.679 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T00:11:40.698 INFO:teuthology.orchestra.run.smithi027.stdout:571230650375 2024-09-15T00:11:40.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T00:11:40.708 INFO:teuthology.orchestra.run.smithi027.stdout:47244640295 2024-09-15T00:11:40.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T00:11:40.709 INFO:teuthology.orchestra.run.smithi027.stdout:506806140938 2024-09-15T00:11:40.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T00:11:40.716 INFO:teuthology.orchestra.run.smithi027.stdout:47244640295 2024-09-15T00:11:40.716 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T00:11:41.057 INFO:teuthology.orchestra.run.smithi027.stdout:47244640294 2024-09-15T00:11:41.069 INFO:teuthology.orchestra.run.smithi027.stdout:506806140937 2024-09-15T00:11:41.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640294 for osd.5 2024-09-15T00:11:41.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 506806140938 got 506806140937 for osd.4 2024-09-15T00:11:41.137 INFO:teuthology.orchestra.run.smithi027.stdout:47244640295 2024-09-15T00:11:41.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640296 got 47244640295 for osd.3 2024-09-15T00:11:41.162 INFO:teuthology.orchestra.run.smithi027.stdout:47244640294 2024-09-15T00:11:41.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640294 for osd.2 2024-09-15T00:11:41.188 INFO:teuthology.orchestra.run.smithi027.stdout:571230650374 2024-09-15T00:11:41.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 571230650375 got 571230650374 for osd.0 2024-09-15T00:11:41.209 INFO:teuthology.orchestra.run.smithi027.stdout:47244640294 2024-09-15T00:11:41.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640294 for osd.1 2024-09-15T00:11:42.072 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T00:11:42.084 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T00:11:42.151 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T00:11:42.178 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T00:11:42.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T00:11:42.224 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T00:11:42.485 INFO:teuthology.orchestra.run.smithi027.stdout:47244640295 2024-09-15T00:11:42.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640295 for osd.5 2024-09-15T00:11:42.499 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:42.511 INFO:teuthology.orchestra.run.smithi027.stdout:47244640296 2024-09-15T00:11:42.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640296 got 47244640296 for osd.3 2024-09-15T00:11:42.525 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:42.572 INFO:teuthology.orchestra.run.smithi027.stdout:506806140938 2024-09-15T00:11:42.580 INFO:teuthology.orchestra.run.smithi027.stdout:571230650375 2024-09-15T00:11:42.586 INFO:tasks.ceph.ceph_manager.ceph:need seq 506806140938 got 506806140938 for osd.4 2024-09-15T00:11:42.586 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:42.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 571230650375 got 571230650375 for osd.0 2024-09-15T00:11:42.598 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:42.649 INFO:teuthology.orchestra.run.smithi027.stdout:47244640295 2024-09-15T00:11:42.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640295 for osd.2 2024-09-15T00:11:42.662 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:42.678 INFO:teuthology.orchestra.run.smithi027.stdout:47244640295 2024-09-15T00:11:42.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640295 got 47244640295 for osd.1 2024-09-15T00:11:42.691 DEBUG:teuthology.parallel:result is None 2024-09-15T00:11:42.691 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T00:11:42.691 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:43.023 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:43.024 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:11:43.035 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T00:11:42.883567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":79,"num_read":255006,"num_read_kb":1398244,"num_write":21899,"num_write_kb":430527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2466,"num_bytes_recovered":93455272,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19793,"ondisk_log_size":19793,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":311692,"kb_used_data":11404,"kb_used_omap":65,"kb_used_meta":300094,"kb_avail":565919348,"statfs":{"total":579820584960,"available":579501412352,"internally_reserved":0,"allocated":11677696,"data_stored":6044160,"data_compressed":18090,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":67028,"internal_metadata":307296812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001328"},"pg_stats":[{"pgid":"3.1f","version":"138'535","reported_seq":11587,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875251+0000","last_change":"2024-09-15T00:11:28.816240+0000","last_active":"2024-09-15T00:11:39.875251+0000","last_peered":"2024-09-15T00:11:39.875251+0000","last_clean":"2024-09-15T00:11:39.875251+0000","last_became_active":"2024-09-15T00:11:28.814325+0000","last_became_peered":"2024-09-15T00:11:28.814325+0000","last_unstale":"2024-09-15T00:11:39.875251+0000","last_undegraded":"2024-09-15T00:11:39.875251+0000","last_fullsized":"2024-09-15T00:11:39.875251+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:19.249372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001206129,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12081,"num_read_kb":64593,"num_write":535,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"135'496","reported_seq":5531,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875628+0000","last_change":"2024-09-15T00:11:16.808902+0000","last_active":"2024-09-15T00:11:39.875628+0000","last_peered":"2024-09-15T00:11:39.875628+0000","last_clean":"2024-09-15T00:11:39.875628+0000","last_became_active":"2024-09-15T00:10:58.347628+0000","last_became_peered":"2024-09-15T00:10:58.347628+0000","last_unstale":"2024-09-15T00:11:39.875628+0000","last_undegraded":"2024-09-15T00:11:39.875628+0000","last_fullsized":"2024-09-15T00:11:39.875628+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:19.680950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014193910000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6067,"num_read_kb":40333,"num_write":376,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"137'826","reported_seq":5711,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876271+0000","last_change":"2024-09-15T00:11:16.819173+0000","last_active":"2024-09-15T00:11:39.876271+0000","last_peered":"2024-09-15T00:11:39.876271+0000","last_clean":"2024-09-15T00:11:39.876271+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:11:39.876271+0000","last_undegraded":"2024-09-15T00:11:39.876271+0000","last_fullsized":"2024-09-15T00:11:39.876271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:38.089567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019789569999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8748,"num_read_kb":37428,"num_write":973,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"138'512","reported_seq":6131,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876263+0000","last_change":"2024-09-15T00:11:16.818860+0000","last_active":"2024-09-15T00:11:39.876263+0000","last_peered":"2024-09-15T00:11:39.876263+0000","last_clean":"2024-09-15T00:11:39.876263+0000","last_became_active":"2024-09-15T00:10:58.347433+0000","last_became_peered":"2024-09-15T00:10:58.347433+0000","last_unstale":"2024-09-15T00:11:39.876263+0000","last_undegraded":"2024-09-15T00:11:39.876263+0000","last_fullsized":"2024-09-15T00:11:39.876263+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:34.872358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878240000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6371,"num_read_kb":39648,"num_write":614,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"137'519","reported_seq":10491,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875469+0000","last_change":"2024-09-15T00:11:27.816441+0000","last_active":"2024-09-15T00:11:39.875469+0000","last_peered":"2024-09-15T00:11:39.875469+0000","last_clean":"2024-09-15T00:11:39.875469+0000","last_became_active":"2024-09-15T00:11:27.814209+0000","last_became_peered":"2024-09-15T00:11:27.814209+0000","last_unstale":"2024-09-15T00:11:39.875469+0000","last_undegraded":"2024-09-15T00:11:39.875469+0000","last_fullsized":"2024-09-15T00:11:39.875469+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:42.793496+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016246954000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11088,"num_read_kb":77879,"num_write":574,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"137'534","reported_seq":11124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797219+0000","last_change":"2024-09-15T00:11:16.807504+0000","last_active":"2024-09-15T00:11:40.797219+0000","last_peered":"2024-09-15T00:11:40.797219+0000","last_clean":"2024-09-15T00:11:40.797219+0000","last_became_active":"2024-09-15T00:10:58.346770+0000","last_became_peered":"2024-09-15T00:10:58.346770+0000","last_unstale":"2024-09-15T00:11:40.797219+0000","last_undegraded":"2024-09-15T00:11:40.797219+0000","last_fullsized":"2024-09-15T00:11:40.797219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:59.341684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015136839999999999,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":13910,"num_read_kb":40588,"num_write":444,"num_write_kb":8754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'709","reported_seq":10797,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457803+0000","last_change":"2024-09-15T00:11:16.811299+0000","last_active":"2024-09-15T00:11:40.457803+0000","last_peered":"2024-09-15T00:11:40.457803+0000","last_clean":"2024-09-15T00:11:40.457803+0000","last_became_active":"2024-09-15T00:10:58.350007+0000","last_became_peered":"2024-09-15T00:10:58.350007+0000","last_unstale":"2024-09-15T00:11:40.457803+0000","last_undegraded":"2024-09-15T00:11:40.457803+0000","last_fullsized":"2024-09-15T00:11:40.457803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:11.344098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019477310000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13597,"num_read_kb":55399,"num_write":535,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"137'655","reported_seq":8927,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875371+0000","last_change":"2024-09-15T00:11:28.815566+0000","last_active":"2024-09-15T00:11:39.875371+0000","last_peered":"2024-09-15T00:11:39.875371+0000","last_clean":"2024-09-15T00:11:39.875371+0000","last_became_active":"2024-09-15T00:11:28.813725+0000","last_became_peered":"2024-09-15T00:11:28.813725+0000","last_unstale":"2024-09-15T00:11:39.875371+0000","last_undegraded":"2024-09-15T00:11:39.875371+0000","last_fullsized":"2024-09-15T00:11:39.875371+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:28.982527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012280349999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9415,"num_read_kb":64591,"num_write":699,"num_write_kb":14239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"136'859","reported_seq":15047,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875360+0000","last_change":"2024-09-15T00:11:27.813449+0000","last_active":"2024-09-15T00:11:39.875360+0000","last_peered":"2024-09-15T00:11:39.875360+0000","last_clean":"2024-09-15T00:11:39.875360+0000","last_became_active":"2024-09-15T00:11:27.810983+0000","last_became_peered":"2024-09-15T00:11:27.810983+0000","last_unstale":"2024-09-15T00:11:39.875360+0000","last_undegraded":"2024-09-15T00:11:39.875360+0000","last_fullsized":"2024-09-15T00:11:39.875360+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:14.048930+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050753860000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":16900,"num_read_kb":72084,"num_write":902,"num_write_kb":44641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'832","reported_seq":8179,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876262+0000","last_change":"2024-09-15T00:11:16.818288+0000","last_active":"2024-09-15T00:11:39.876262+0000","last_peered":"2024-09-15T00:11:39.876262+0000","last_clean":"2024-09-15T00:11:39.876262+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:11:39.876262+0000","last_undegraded":"2024-09-15T00:11:39.876262+0000","last_fullsized":"2024-09-15T00:11:39.876262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.421381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098720300000000008,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10216,"num_read_kb":33960,"num_write":980,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"136'460","reported_seq":7066,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461827+0000","last_change":"2024-09-15T00:11:27.811672+0000","last_active":"2024-09-15T00:11:40.461827+0000","last_peered":"2024-09-15T00:11:40.461827+0000","last_clean":"2024-09-15T00:11:40.461827+0000","last_became_active":"2024-09-15T00:11:27.810534+0000","last_became_peered":"2024-09-15T00:11:27.810534+0000","last_unstale":"2024-09-15T00:11:40.461827+0000","last_undegraded":"2024-09-15T00:11:40.461827+0000","last_fullsized":"2024-09-15T00:11:40.461827+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:22:56.578888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081705799999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6309,"num_read_kb":62113,"num_write":514,"num_write_kb":29400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"136'577","reported_seq":8459,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461730+0000","last_change":"2024-09-15T00:11:16.822160+0000","last_active":"2024-09-15T00:11:40.461730+0000","last_peered":"2024-09-15T00:11:40.461730+0000","last_clean":"2024-09-15T00:11:40.461730+0000","last_became_active":"2024-09-15T00:10:58.353801+0000","last_became_peered":"2024-09-15T00:10:58.353801+0000","last_unstale":"2024-09-15T00:11:40.461730+0000","last_undegraded":"2024-09-15T00:11:40.461730+0000","last_fullsized":"2024-09-15T00:11:40.461730+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:56:52.432800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503938,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9435,"num_read_kb":45249,"num_write":397,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":310,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461963+0000","last_change":"2024-09-15T00:10:58.352604+0000","last_active":"2024-09-15T00:11:40.461963+0000","last_peered":"2024-09-15T00:11:40.461963+0000","last_clean":"2024-09-15T00:11:40.461963+0000","last_became_active":"2024-09-15T00:10:58.352314+0000","last_became_peered":"2024-09-15T00:10:58.352314+0000","last_unstale":"2024-09-15T00:11:40.461963+0000","last_undegraded":"2024-09-15T00:11:40.461963+0000","last_fullsized":"2024-09-15T00:11:40.461963+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:11.117426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"138'455","reported_seq":3568,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875353+0000","last_change":"2024-09-15T00:11:26.812244+0000","last_active":"2024-09-15T00:11:39.875353+0000","last_peered":"2024-09-15T00:11:39.875353+0000","last_clean":"2024-09-15T00:11:39.875353+0000","last_became_active":"2024-09-15T00:11:26.810222+0000","last_became_peered":"2024-09-15T00:11:26.810222+0000","last_unstale":"2024-09-15T00:11:39.875353+0000","last_undegraded":"2024-09-15T00:11:39.875353+0000","last_fullsized":"2024-09-15T00:11:39.875353+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:04:31.939139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099603400000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2834,"num_read_kb":21154,"num_write":404,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":302,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458045+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:11:40.458045+0000","last_peered":"2024-09-15T00:11:40.458045+0000","last_clean":"2024-09-15T00:11:40.458045+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:11:40.458045+0000","last_undegraded":"2024-09-15T00:11:40.458045+0000","last_fullsized":"2024-09-15T00:11:40.458045+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'301","reported_seq":3604,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808663+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:52.093302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3137,"num_read_kb":36813,"num_write":322,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461876+0000","last_change":"2024-09-15T00:11:23.821084+0000","last_active":"2024-09-15T00:11:40.461876+0000","last_peered":"2024-09-15T00:11:40.461876+0000","last_clean":"2024-09-15T00:11:40.461876+0000","last_became_active":"2024-09-15T00:11:23.820814+0000","last_became_peered":"2024-09-15T00:11:23.820814+0000","last_unstale":"2024-09-15T00:11:40.461876+0000","last_undegraded":"2024-09-15T00:11:40.461876+0000","last_fullsized":"2024-09-15T00:11:40.461876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:05:01.106813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"138'804","reported_seq":7267,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457816+0000","last_change":"2024-09-15T00:11:16.810988+0000","last_active":"2024-09-15T00:11:40.457816+0000","last_peered":"2024-09-15T00:11:40.457816+0000","last_clean":"2024-09-15T00:11:40.457816+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:11:40.457816+0000","last_undegraded":"2024-09-15T00:11:40.457816+0000","last_fullsized":"2024-09-15T00:11:40.457816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:19:00.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015707939999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9943,"num_read_kb":67757,"num_write":840,"num_write_kb":50219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":321,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457701+0000","last_change":"2024-09-15T00:11:23.823985+0000","last_active":"2024-09-15T00:11:40.457701+0000","last_peered":"2024-09-15T00:11:40.457701+0000","last_clean":"2024-09-15T00:11:40.457701+0000","last_became_active":"2024-09-15T00:11:23.823594+0000","last_became_peered":"2024-09-15T00:11:23.823594+0000","last_unstale":"2024-09-15T00:11:40.457701+0000","last_undegraded":"2024-09-15T00:11:40.457701+0000","last_fullsized":"2024-09-15T00:11:40.457701+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:01:04.517222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'598","reported_seq":5986,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876312+0000","last_change":"2024-09-15T00:11:16.819731+0000","last_active":"2024-09-15T00:11:39.876312+0000","last_peered":"2024-09-15T00:11:39.876312+0000","last_clean":"2024-09-15T00:11:39.876312+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:11:39.876312+0000","last_undegraded":"2024-09-15T00:11:39.876312+0000","last_fullsized":"2024-09-15T00:11:39.876312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:56.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022079199999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6563,"num_read_kb":34828,"num_write":657,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"138'669","reported_seq":6107,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797327+0000","last_change":"2024-09-15T00:11:26.813300+0000","last_active":"2024-09-15T00:11:40.797327+0000","last_peered":"2024-09-15T00:11:40.797327+0000","last_clean":"2024-09-15T00:11:40.797327+0000","last_became_active":"2024-09-15T00:11:26.812061+0000","last_became_peered":"2024-09-15T00:11:26.812061+0000","last_unstale":"2024-09-15T00:11:40.797327+0000","last_undegraded":"2024-09-15T00:11:40.797327+0000","last_fullsized":"2024-09-15T00:11:40.797327+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:59:00.972440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013113269999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7232,"num_read_kb":26564,"num_write":565,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876199+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:11:39.876199+0000","last_peered":"2024-09-15T00:11:39.876199+0000","last_clean":"2024-09-15T00:11:39.876199+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:11:39.876199+0000","last_undegraded":"2024-09-15T00:11:39.876199+0000","last_fullsized":"2024-09-15T00:11:39.876199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"136'714","reported_seq":7159,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457702+0000","last_change":"2024-09-15T00:11:16.811546+0000","last_active":"2024-09-15T00:11:40.457702+0000","last_peered":"2024-09-15T00:11:40.457702+0000","last_clean":"2024-09-15T00:11:40.457702+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:11:40.457702+0000","last_undegraded":"2024-09-15T00:11:40.457702+0000","last_fullsized":"2024-09-15T00:11:40.457702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:46.467190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570948,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7803,"num_read_kb":54109,"num_write":924,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875605+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:11:39.875605+0000","last_peered":"2024-09-15T00:11:39.875605+0000","last_clean":"2024-09-15T00:11:39.875605+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:11:39.875605+0000","last_undegraded":"2024-09-15T00:11:39.875605+0000","last_fullsized":"2024-09-15T00:11:39.875605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'627","reported_seq":3714,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875090+0000","last_change":"2024-09-15T00:11:26.815022+0000","last_active":"2024-09-15T00:11:39.875090+0000","last_peered":"2024-09-15T00:11:39.875090+0000","last_clean":"2024-09-15T00:11:39.875090+0000","last_became_active":"2024-09-15T00:11:26.813953+0000","last_became_peered":"2024-09-15T00:11:26.813953+0000","last_unstale":"2024-09-15T00:11:39.875090+0000","last_undegraded":"2024-09-15T00:11:39.875090+0000","last_fullsized":"2024-09-15T00:11:39.875090+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:43:46.140096+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00071073200000000005,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5290,"num_read_kb":27132,"num_write":878,"num_write_kb":8037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2365199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":303,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461630+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:11:40.461630+0000","last_peered":"2024-09-15T00:11:40.461630+0000","last_clean":"2024-09-15T00:11:40.461630+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:11:40.461630+0000","last_undegraded":"2024-09-15T00:11:40.461630+0000","last_fullsized":"2024-09-15T00:11:40.461630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'493","reported_seq":6880,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458109+0000","last_change":"2024-09-15T00:11:16.810764+0000","last_active":"2024-09-15T00:11:40.458109+0000","last_peered":"2024-09-15T00:11:40.458109+0000","last_clean":"2024-09-15T00:11:40.458109+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:11:40.458109+0000","last_undegraded":"2024-09-15T00:11:40.458109+0000","last_fullsized":"2024-09-15T00:11:40.458109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:10:05.221091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215973,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6232,"num_read_kb":25476,"num_write":407,"num_write_kb":5284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":305,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876278+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:11:39.876278+0000","last_peered":"2024-09-15T00:11:39.876278+0000","last_clean":"2024-09-15T00:11:39.876278+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:11:39.876278+0000","last_undegraded":"2024-09-15T00:11:39.876278+0000","last_fullsized":"2024-09-15T00:11:39.876278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":482,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461680+0000","last_change":"2024-09-15T00:11:26.809502+0000","last_active":"2024-09-15T00:11:40.461680+0000","last_peered":"2024-09-15T00:11:40.461680+0000","last_clean":"2024-09-15T00:11:40.461680+0000","last_became_active":"2024-09-15T00:11:26.809081+0000","last_became_peered":"2024-09-15T00:11:26.809081+0000","last_unstale":"2024-09-15T00:11:40.461680+0000","last_undegraded":"2024-09-15T00:11:40.461680+0000","last_fullsized":"2024-09-15T00:11:40.461680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:33.161918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"136'570","reported_seq":11116,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808250+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:18.362309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098862600000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":12740,"num_read_kb":88117,"num_write":607,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"137'511","reported_seq":9033,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797237+0000","last_change":"2024-09-15T00:11:16.806418+0000","last_active":"2024-09-15T00:11:40.797237+0000","last_peered":"2024-09-15T00:11:40.797237+0000","last_clean":"2024-09-15T00:11:40.797237+0000","last_became_active":"2024-09-15T00:10:58.346994+0000","last_became_peered":"2024-09-15T00:10:58.346994+0000","last_unstale":"2024-09-15T00:11:40.797237+0000","last_undegraded":"2024-09-15T00:11:40.797237+0000","last_fullsized":"2024-09-15T00:11:40.797237+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:09:39.692848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088599499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9287,"num_read_kb":49680,"num_write":419,"num_write_kb":13143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"137'346","reported_seq":2534,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461768+0000","last_change":"2024-09-15T00:11:16.822630+0000","last_active":"2024-09-15T00:11:40.461768+0000","last_peered":"2024-09-15T00:11:40.461768+0000","last_clean":"2024-09-15T00:11:40.461768+0000","last_became_active":"2024-09-15T00:10:58.353160+0000","last_became_peered":"2024-09-15T00:10:58.353160+0000","last_unstale":"2024-09-15T00:11:40.461768+0000","last_undegraded":"2024-09-15T00:11:40.461768+0000","last_fullsized":"2024-09-15T00:11:40.461768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:32.587056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001573252,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2498,"num_read_kb":16284,"num_write":316,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"137'342","reported_seq":4084,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457821+0000","last_change":"2024-09-15T00:11:27.813271+0000","last_active":"2024-09-15T00:11:40.457821+0000","last_peered":"2024-09-15T00:11:40.457821+0000","last_clean":"2024-09-15T00:11:40.457821+0000","last_became_active":"2024-09-15T00:11:27.812092+0000","last_became_peered":"2024-09-15T00:11:27.812092+0000","last_unstale":"2024-09-15T00:11:40.457821+0000","last_undegraded":"2024-09-15T00:11:40.457821+0000","last_fullsized":"2024-09-15T00:11:40.457821+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:10:35.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000721333,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3470,"num_read_kb":33354,"num_write":301,"num_write_kb":6941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"138'1089","reported_seq":6155,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876331+0000","last_change":"2024-09-15T00:11:27.812382+0000","last_active":"2024-09-15T00:11:39.876331+0000","last_peered":"2024-09-15T00:11:39.876331+0000","last_clean":"2024-09-15T00:11:39.876331+0000","last_became_active":"2024-09-15T00:11:27.811047+0000","last_became_peered":"2024-09-15T00:11:27.811047+0000","last_unstale":"2024-09-15T00:11:39.876331+0000","last_undegraded":"2024-09-15T00:11:39.876331+0000","last_fullsized":"2024-09-15T00:11:39.876331+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1089,"log_dups_size":0,"ondisk_log_size":1089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:34.938354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021428929999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8751,"num_read_kb":31066,"num_write":1650,"num_write_kb":11591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'472","reported_seq":5184,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461560+0000","last_change":"2024-09-15T00:11:16.821809+0000","last_active":"2024-09-15T00:11:40.461560+0000","last_peered":"2024-09-15T00:11:40.461560+0000","last_clean":"2024-09-15T00:11:40.461560+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:11:40.461560+0000","last_undegraded":"2024-09-15T00:11:40.461560+0000","last_fullsized":"2024-09-15T00:11:40.461560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:55:03.400561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012186440000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6281,"num_read_kb":25421,"num_write":438,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"137'321","reported_seq":4493,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876475+0000","last_change":"2024-09-15T00:11:16.818687+0000","last_active":"2024-09-15T00:11:39.876475+0000","last_peered":"2024-09-15T00:11:39.876475+0000","last_clean":"2024-09-15T00:11:39.876475+0000","last_became_active":"2024-09-15T00:10:58.347480+0000","last_became_peered":"2024-09-15T00:10:58.347480+0000","last_unstale":"2024-09-15T00:11:39.876475+0000","last_undegraded":"2024-09-15T00:11:39.876475+0000","last_fullsized":"2024-09-15T00:11:39.876475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:57.538275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015201839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3806,"num_read_kb":22330,"num_write":293,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"135'777","reported_seq":5222,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876408+0000","last_change":"2024-09-15T00:11:16.819308+0000","last_active":"2024-09-15T00:11:39.876408+0000","last_peered":"2024-09-15T00:11:39.876408+0000","last_clean":"2024-09-15T00:11:39.876408+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:11:39.876408+0000","last_undegraded":"2024-09-15T00:11:39.876408+0000","last_fullsized":"2024-09-15T00:11:39.876408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:38.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018611719999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7063,"num_read_kb":22283,"num_write":819,"num_write_kb":6499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"136'345","reported_seq":6247,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876066+0000","last_change":"2024-09-15T00:11:26.810552+0000","last_active":"2024-09-15T00:11:39.876066+0000","last_peered":"2024-09-15T00:11:39.876066+0000","last_clean":"2024-09-15T00:11:39.876066+0000","last_became_active":"2024-09-15T00:11:26.809733+0000","last_became_peered":"2024-09-15T00:11:26.809733+0000","last_unstale":"2024-09-15T00:11:39.876066+0000","last_undegraded":"2024-09-15T00:11:39.876066+0000","last_fullsized":"2024-09-15T00:11:39.876066+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:03.405500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014647,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5082,"num_read_kb":32663,"num_write":383,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"134'1204","reported_seq":4832,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461699+0000","last_change":"2024-09-15T00:11:16.821784+0000","last_active":"2024-09-15T00:11:40.461699+0000","last_peered":"2024-09-15T00:11:40.461699+0000","last_clean":"2024-09-15T00:11:40.461699+0000","last_became_active":"2024-09-15T00:10:58.354658+0000","last_became_peered":"2024-09-15T00:10:58.354658+0000","last_unstale":"2024-09-15T00:11:40.461699+0000","last_undegraded":"2024-09-15T00:11:40.461699+0000","last_fullsized":"2024-09-15T00:11:40.461699+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:36.660704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185029,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7974,"num_read_kb":21277,"num_write":1815,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'1278","reported_seq":10322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875220+0000","last_change":"2024-09-15T00:11:27.815741+0000","last_active":"2024-09-15T00:11:39.875220+0000","last_peered":"2024-09-15T00:11:39.875220+0000","last_clean":"2024-09-15T00:11:39.875220+0000","last_became_active":"2024-09-15T00:11:27.813284+0000","last_became_peered":"2024-09-15T00:11:27.813284+0000","last_unstale":"2024-09-15T00:11:39.875220+0000","last_undegraded":"2024-09-15T00:11:39.875220+0000","last_fullsized":"2024-09-15T00:11:39.875220+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:58:44.187713+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010772937999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":12900,"num_read_kb":112467,"num_write":1928,"num_write_kb":43546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"137'243","reported_seq":2506,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461549+0000","last_change":"2024-09-15T00:11:16.822816+0000","last_active":"2024-09-15T00:11:40.461549+0000","last_peered":"2024-09-15T00:11:40.461549+0000","last_clean":"2024-09-15T00:11:40.461549+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:11:40.461549+0000","last_undegraded":"2024-09-15T00:11:40.461549+0000","last_fullsized":"2024-09-15T00:11:40.461549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:36:47.914784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013592509999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1867,"num_read_kb":15504,"num_write":229,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":79,"num_read":254890,"num_read_kb":1398144,"num_write":21738,"num_write_kb":428109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2462,"num_bytes_recovered":92274536,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26303,"internal_metadata":0},"log_size":19673,"ondisk_log_size":19673,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":27652,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59444,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312396,"statfs":{"total":96636764160,"available":96575893504,"internally_reserved":0,"allocated":1933312,"data_stored":811415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":118,"seq":506806140939,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34284,"kb_used_data":1876,"kb_used_omap":31,"kb_used_meta":32352,"kb_avail":94337556,"statfs":{"total":96636764160,"available":96601657344,"internally_reserved":0,"allocated":1921024,"data_stored":806387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31754,"internal_metadata":33129462},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81704,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290136,"statfs":{"total":96636764160,"available":96553099264,"internally_reserved":0,"allocated":1986560,"data_stored":1401764,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640296,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":1929216,"data_stored":811396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":811434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":133,"seq":571230650376,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39580,"kb_used_data":1924,"kb_used_omap":28,"kb_used_meta":37603,"kb_avail":94332260,"statfs":{"total":96636764160,"available":96596234240,"internally_reserved":0,"allocated":1970176,"data_stored":1401764,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28896,"internal_metadata":38506272},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15263,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11040,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:43.040 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:43.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:43.329 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:11:43.341 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T00:11:42.883567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":79,"num_read":255006,"num_read_kb":1398244,"num_write":21899,"num_write_kb":430527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2466,"num_bytes_recovered":93455272,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19793,"ondisk_log_size":19793,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":311692,"kb_used_data":11404,"kb_used_omap":65,"kb_used_meta":300094,"kb_avail":565919348,"statfs":{"total":579820584960,"available":579501412352,"internally_reserved":0,"allocated":11677696,"data_stored":6044160,"data_compressed":18090,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":67028,"internal_metadata":307296812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001328"},"pg_stats":[{"pgid":"3.1f","version":"138'535","reported_seq":11587,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875251+0000","last_change":"2024-09-15T00:11:28.816240+0000","last_active":"2024-09-15T00:11:39.875251+0000","last_peered":"2024-09-15T00:11:39.875251+0000","last_clean":"2024-09-15T00:11:39.875251+0000","last_became_active":"2024-09-15T00:11:28.814325+0000","last_became_peered":"2024-09-15T00:11:28.814325+0000","last_unstale":"2024-09-15T00:11:39.875251+0000","last_undegraded":"2024-09-15T00:11:39.875251+0000","last_fullsized":"2024-09-15T00:11:39.875251+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:19.249372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001206129,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12081,"num_read_kb":64593,"num_write":535,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"135'496","reported_seq":5531,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875628+0000","last_change":"2024-09-15T00:11:16.808902+0000","last_active":"2024-09-15T00:11:39.875628+0000","last_peered":"2024-09-15T00:11:39.875628+0000","last_clean":"2024-09-15T00:11:39.875628+0000","last_became_active":"2024-09-15T00:10:58.347628+0000","last_became_peered":"2024-09-15T00:10:58.347628+0000","last_unstale":"2024-09-15T00:11:39.875628+0000","last_undegraded":"2024-09-15T00:11:39.875628+0000","last_fullsized":"2024-09-15T00:11:39.875628+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:19.680950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014193910000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6067,"num_read_kb":40333,"num_write":376,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"137'826","reported_seq":5711,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876271+0000","last_change":"2024-09-15T00:11:16.819173+0000","last_active":"2024-09-15T00:11:39.876271+0000","last_peered":"2024-09-15T00:11:39.876271+0000","last_clean":"2024-09-15T00:11:39.876271+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:11:39.876271+0000","last_undegraded":"2024-09-15T00:11:39.876271+0000","last_fullsized":"2024-09-15T00:11:39.876271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:38.089567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019789569999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8748,"num_read_kb":37428,"num_write":973,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"138'512","reported_seq":6131,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876263+0000","last_change":"2024-09-15T00:11:16.818860+0000","last_active":"2024-09-15T00:11:39.876263+0000","last_peered":"2024-09-15T00:11:39.876263+0000","last_clean":"2024-09-15T00:11:39.876263+0000","last_became_active":"2024-09-15T00:10:58.347433+0000","last_became_peered":"2024-09-15T00:10:58.347433+0000","last_unstale":"2024-09-15T00:11:39.876263+0000","last_undegraded":"2024-09-15T00:11:39.876263+0000","last_fullsized":"2024-09-15T00:11:39.876263+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:34.872358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878240000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6371,"num_read_kb":39648,"num_write":614,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"137'519","reported_seq":10491,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875469+0000","last_change":"2024-09-15T00:11:27.816441+0000","last_active":"2024-09-15T00:11:39.875469+0000","last_peered":"2024-09-15T00:11:39.875469+0000","last_clean":"2024-09-15T00:11:39.875469+0000","last_became_active":"2024-09-15T00:11:27.814209+0000","last_became_peered":"2024-09-15T00:11:27.814209+0000","last_unstale":"2024-09-15T00:11:39.875469+0000","last_undegraded":"2024-09-15T00:11:39.875469+0000","last_fullsized":"2024-09-15T00:11:39.875469+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:42.793496+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016246954000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11088,"num_read_kb":77879,"num_write":574,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"137'534","reported_seq":11124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797219+0000","last_change":"2024-09-15T00:11:16.807504+0000","last_active":"2024-09-15T00:11:40.797219+0000","last_peered":"2024-09-15T00:11:40.797219+0000","last_clean":"2024-09-15T00:11:40.797219+0000","last_became_active":"2024-09-15T00:10:58.346770+0000","last_became_peered":"2024-09-15T00:10:58.346770+0000","last_unstale":"2024-09-15T00:11:40.797219+0000","last_undegraded":"2024-09-15T00:11:40.797219+0000","last_fullsized":"2024-09-15T00:11:40.797219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:59.341684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015136839999999999,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":13910,"num_read_kb":40588,"num_write":444,"num_write_kb":8754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'709","reported_seq":10797,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457803+0000","last_change":"2024-09-15T00:11:16.811299+0000","last_active":"2024-09-15T00:11:40.457803+0000","last_peered":"2024-09-15T00:11:40.457803+0000","last_clean":"2024-09-15T00:11:40.457803+0000","last_became_active":"2024-09-15T00:10:58.350007+0000","last_became_peered":"2024-09-15T00:10:58.350007+0000","last_unstale":"2024-09-15T00:11:40.457803+0000","last_undegraded":"2024-09-15T00:11:40.457803+0000","last_fullsized":"2024-09-15T00:11:40.457803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:11.344098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019477310000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13597,"num_read_kb":55399,"num_write":535,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"137'655","reported_seq":8927,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875371+0000","last_change":"2024-09-15T00:11:28.815566+0000","last_active":"2024-09-15T00:11:39.875371+0000","last_peered":"2024-09-15T00:11:39.875371+0000","last_clean":"2024-09-15T00:11:39.875371+0000","last_became_active":"2024-09-15T00:11:28.813725+0000","last_became_peered":"2024-09-15T00:11:28.813725+0000","last_unstale":"2024-09-15T00:11:39.875371+0000","last_undegraded":"2024-09-15T00:11:39.875371+0000","last_fullsized":"2024-09-15T00:11:39.875371+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:28.982527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012280349999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9415,"num_read_kb":64591,"num_write":699,"num_write_kb":14239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"136'859","reported_seq":15047,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875360+0000","last_change":"2024-09-15T00:11:27.813449+0000","last_active":"2024-09-15T00:11:39.875360+0000","last_peered":"2024-09-15T00:11:39.875360+0000","last_clean":"2024-09-15T00:11:39.875360+0000","last_became_active":"2024-09-15T00:11:27.810983+0000","last_became_peered":"2024-09-15T00:11:27.810983+0000","last_unstale":"2024-09-15T00:11:39.875360+0000","last_undegraded":"2024-09-15T00:11:39.875360+0000","last_fullsized":"2024-09-15T00:11:39.875360+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:14.048930+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050753860000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":16900,"num_read_kb":72084,"num_write":902,"num_write_kb":44641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'832","reported_seq":8179,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876262+0000","last_change":"2024-09-15T00:11:16.818288+0000","last_active":"2024-09-15T00:11:39.876262+0000","last_peered":"2024-09-15T00:11:39.876262+0000","last_clean":"2024-09-15T00:11:39.876262+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:11:39.876262+0000","last_undegraded":"2024-09-15T00:11:39.876262+0000","last_fullsized":"2024-09-15T00:11:39.876262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.421381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098720300000000008,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10216,"num_read_kb":33960,"num_write":980,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"136'460","reported_seq":7066,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461827+0000","last_change":"2024-09-15T00:11:27.811672+0000","last_active":"2024-09-15T00:11:40.461827+0000","last_peered":"2024-09-15T00:11:40.461827+0000","last_clean":"2024-09-15T00:11:40.461827+0000","last_became_active":"2024-09-15T00:11:27.810534+0000","last_became_peered":"2024-09-15T00:11:27.810534+0000","last_unstale":"2024-09-15T00:11:40.461827+0000","last_undegraded":"2024-09-15T00:11:40.461827+0000","last_fullsized":"2024-09-15T00:11:40.461827+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:22:56.578888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081705799999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6309,"num_read_kb":62113,"num_write":514,"num_write_kb":29400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"136'577","reported_seq":8459,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461730+0000","last_change":"2024-09-15T00:11:16.822160+0000","last_active":"2024-09-15T00:11:40.461730+0000","last_peered":"2024-09-15T00:11:40.461730+0000","last_clean":"2024-09-15T00:11:40.461730+0000","last_became_active":"2024-09-15T00:10:58.353801+0000","last_became_peered":"2024-09-15T00:10:58.353801+0000","last_unstale":"2024-09-15T00:11:40.461730+0000","last_undegraded":"2024-09-15T00:11:40.461730+0000","last_fullsized":"2024-09-15T00:11:40.461730+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:56:52.432800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503938,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9435,"num_read_kb":45249,"num_write":397,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":310,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461963+0000","last_change":"2024-09-15T00:10:58.352604+0000","last_active":"2024-09-15T00:11:40.461963+0000","last_peered":"2024-09-15T00:11:40.461963+0000","last_clean":"2024-09-15T00:11:40.461963+0000","last_became_active":"2024-09-15T00:10:58.352314+0000","last_became_peered":"2024-09-15T00:10:58.352314+0000","last_unstale":"2024-09-15T00:11:40.461963+0000","last_undegraded":"2024-09-15T00:11:40.461963+0000","last_fullsized":"2024-09-15T00:11:40.461963+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:11.117426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"138'455","reported_seq":3568,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875353+0000","last_change":"2024-09-15T00:11:26.812244+0000","last_active":"2024-09-15T00:11:39.875353+0000","last_peered":"2024-09-15T00:11:39.875353+0000","last_clean":"2024-09-15T00:11:39.875353+0000","last_became_active":"2024-09-15T00:11:26.810222+0000","last_became_peered":"2024-09-15T00:11:26.810222+0000","last_unstale":"2024-09-15T00:11:39.875353+0000","last_undegraded":"2024-09-15T00:11:39.875353+0000","last_fullsized":"2024-09-15T00:11:39.875353+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:04:31.939139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099603400000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2834,"num_read_kb":21154,"num_write":404,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":302,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458045+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:11:40.458045+0000","last_peered":"2024-09-15T00:11:40.458045+0000","last_clean":"2024-09-15T00:11:40.458045+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:11:40.458045+0000","last_undegraded":"2024-09-15T00:11:40.458045+0000","last_fullsized":"2024-09-15T00:11:40.458045+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'301","reported_seq":3604,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808663+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:52.093302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3137,"num_read_kb":36813,"num_write":322,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461876+0000","last_change":"2024-09-15T00:11:23.821084+0000","last_active":"2024-09-15T00:11:40.461876+0000","last_peered":"2024-09-15T00:11:40.461876+0000","last_clean":"2024-09-15T00:11:40.461876+0000","last_became_active":"2024-09-15T00:11:23.820814+0000","last_became_peered":"2024-09-15T00:11:23.820814+0000","last_unstale":"2024-09-15T00:11:40.461876+0000","last_undegraded":"2024-09-15T00:11:40.461876+0000","last_fullsized":"2024-09-15T00:11:40.461876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:05:01.106813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"138'804","reported_seq":7267,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457816+0000","last_change":"2024-09-15T00:11:16.810988+0000","last_active":"2024-09-15T00:11:40.457816+0000","last_peered":"2024-09-15T00:11:40.457816+0000","last_clean":"2024-09-15T00:11:40.457816+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:11:40.457816+0000","last_undegraded":"2024-09-15T00:11:40.457816+0000","last_fullsized":"2024-09-15T00:11:40.457816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:19:00.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015707939999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9943,"num_read_kb":67757,"num_write":840,"num_write_kb":50219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":321,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457701+0000","last_change":"2024-09-15T00:11:23.823985+0000","last_active":"2024-09-15T00:11:40.457701+0000","last_peered":"2024-09-15T00:11:40.457701+0000","last_clean":"2024-09-15T00:11:40.457701+0000","last_became_active":"2024-09-15T00:11:23.823594+0000","last_became_peered":"2024-09-15T00:11:23.823594+0000","last_unstale":"2024-09-15T00:11:40.457701+0000","last_undegraded":"2024-09-15T00:11:40.457701+0000","last_fullsized":"2024-09-15T00:11:40.457701+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:01:04.517222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'598","reported_seq":5986,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876312+0000","last_change":"2024-09-15T00:11:16.819731+0000","last_active":"2024-09-15T00:11:39.876312+0000","last_peered":"2024-09-15T00:11:39.876312+0000","last_clean":"2024-09-15T00:11:39.876312+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:11:39.876312+0000","last_undegraded":"2024-09-15T00:11:39.876312+0000","last_fullsized":"2024-09-15T00:11:39.876312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:56.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022079199999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6563,"num_read_kb":34828,"num_write":657,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"138'669","reported_seq":6107,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797327+0000","last_change":"2024-09-15T00:11:26.813300+0000","last_active":"2024-09-15T00:11:40.797327+0000","last_peered":"2024-09-15T00:11:40.797327+0000","last_clean":"2024-09-15T00:11:40.797327+0000","last_became_active":"2024-09-15T00:11:26.812061+0000","last_became_peered":"2024-09-15T00:11:26.812061+0000","last_unstale":"2024-09-15T00:11:40.797327+0000","last_undegraded":"2024-09-15T00:11:40.797327+0000","last_fullsized":"2024-09-15T00:11:40.797327+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:59:00.972440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013113269999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7232,"num_read_kb":26564,"num_write":565,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876199+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:11:39.876199+0000","last_peered":"2024-09-15T00:11:39.876199+0000","last_clean":"2024-09-15T00:11:39.876199+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:11:39.876199+0000","last_undegraded":"2024-09-15T00:11:39.876199+0000","last_fullsized":"2024-09-15T00:11:39.876199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"136'714","reported_seq":7159,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457702+0000","last_change":"2024-09-15T00:11:16.811546+0000","last_active":"2024-09-15T00:11:40.457702+0000","last_peered":"2024-09-15T00:11:40.457702+0000","last_clean":"2024-09-15T00:11:40.457702+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:11:40.457702+0000","last_undegraded":"2024-09-15T00:11:40.457702+0000","last_fullsized":"2024-09-15T00:11:40.457702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:46.467190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570948,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7803,"num_read_kb":54109,"num_write":924,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875605+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:11:39.875605+0000","last_peered":"2024-09-15T00:11:39.875605+0000","last_clean":"2024-09-15T00:11:39.875605+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:11:39.875605+0000","last_undegraded":"2024-09-15T00:11:39.875605+0000","last_fullsized":"2024-09-15T00:11:39.875605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'627","reported_seq":3714,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875090+0000","last_change":"2024-09-15T00:11:26.815022+0000","last_active":"2024-09-15T00:11:39.875090+0000","last_peered":"2024-09-15T00:11:39.875090+0000","last_clean":"2024-09-15T00:11:39.875090+0000","last_became_active":"2024-09-15T00:11:26.813953+0000","last_became_peered":"2024-09-15T00:11:26.813953+0000","last_unstale":"2024-09-15T00:11:39.875090+0000","last_undegraded":"2024-09-15T00:11:39.875090+0000","last_fullsized":"2024-09-15T00:11:39.875090+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:43:46.140096+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00071073200000000005,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5290,"num_read_kb":27132,"num_write":878,"num_write_kb":8037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2365199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":303,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461630+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:11:40.461630+0000","last_peered":"2024-09-15T00:11:40.461630+0000","last_clean":"2024-09-15T00:11:40.461630+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:11:40.461630+0000","last_undegraded":"2024-09-15T00:11:40.461630+0000","last_fullsized":"2024-09-15T00:11:40.461630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'493","reported_seq":6880,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458109+0000","last_change":"2024-09-15T00:11:16.810764+0000","last_active":"2024-09-15T00:11:40.458109+0000","last_peered":"2024-09-15T00:11:40.458109+0000","last_clean":"2024-09-15T00:11:40.458109+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:11:40.458109+0000","last_undegraded":"2024-09-15T00:11:40.458109+0000","last_fullsized":"2024-09-15T00:11:40.458109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:10:05.221091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215973,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6232,"num_read_kb":25476,"num_write":407,"num_write_kb":5284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":305,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876278+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:11:39.876278+0000","last_peered":"2024-09-15T00:11:39.876278+0000","last_clean":"2024-09-15T00:11:39.876278+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:11:39.876278+0000","last_undegraded":"2024-09-15T00:11:39.876278+0000","last_fullsized":"2024-09-15T00:11:39.876278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":482,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461680+0000","last_change":"2024-09-15T00:11:26.809502+0000","last_active":"2024-09-15T00:11:40.461680+0000","last_peered":"2024-09-15T00:11:40.461680+0000","last_clean":"2024-09-15T00:11:40.461680+0000","last_became_active":"2024-09-15T00:11:26.809081+0000","last_became_peered":"2024-09-15T00:11:26.809081+0000","last_unstale":"2024-09-15T00:11:40.461680+0000","last_undegraded":"2024-09-15T00:11:40.461680+0000","last_fullsized":"2024-09-15T00:11:40.461680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:33.161918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"136'570","reported_seq":11116,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808250+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:18.362309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098862600000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":12740,"num_read_kb":88117,"num_write":607,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"137'511","reported_seq":9033,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797237+0000","last_change":"2024-09-15T00:11:16.806418+0000","last_active":"2024-09-15T00:11:40.797237+0000","last_peered":"2024-09-15T00:11:40.797237+0000","last_clean":"2024-09-15T00:11:40.797237+0000","last_became_active":"2024-09-15T00:10:58.346994+0000","last_became_peered":"2024-09-15T00:10:58.346994+0000","last_unstale":"2024-09-15T00:11:40.797237+0000","last_undegraded":"2024-09-15T00:11:40.797237+0000","last_fullsized":"2024-09-15T00:11:40.797237+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:09:39.692848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088599499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9287,"num_read_kb":49680,"num_write":419,"num_write_kb":13143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"137'346","reported_seq":2534,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461768+0000","last_change":"2024-09-15T00:11:16.822630+0000","last_active":"2024-09-15T00:11:40.461768+0000","last_peered":"2024-09-15T00:11:40.461768+0000","last_clean":"2024-09-15T00:11:40.461768+0000","last_became_active":"2024-09-15T00:10:58.353160+0000","last_became_peered":"2024-09-15T00:10:58.353160+0000","last_unstale":"2024-09-15T00:11:40.461768+0000","last_undegraded":"2024-09-15T00:11:40.461768+0000","last_fullsized":"2024-09-15T00:11:40.461768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:32.587056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001573252,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2498,"num_read_kb":16284,"num_write":316,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"137'342","reported_seq":4084,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457821+0000","last_change":"2024-09-15T00:11:27.813271+0000","last_active":"2024-09-15T00:11:40.457821+0000","last_peered":"2024-09-15T00:11:40.457821+0000","last_clean":"2024-09-15T00:11:40.457821+0000","last_became_active":"2024-09-15T00:11:27.812092+0000","last_became_peered":"2024-09-15T00:11:27.812092+0000","last_unstale":"2024-09-15T00:11:40.457821+0000","last_undegraded":"2024-09-15T00:11:40.457821+0000","last_fullsized":"2024-09-15T00:11:40.457821+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:10:35.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000721333,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3470,"num_read_kb":33354,"num_write":301,"num_write_kb":6941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"138'1089","reported_seq":6155,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876331+0000","last_change":"2024-09-15T00:11:27.812382+0000","last_active":"2024-09-15T00:11:39.876331+0000","last_peered":"2024-09-15T00:11:39.876331+0000","last_clean":"2024-09-15T00:11:39.876331+0000","last_became_active":"2024-09-15T00:11:27.811047+0000","last_became_peered":"2024-09-15T00:11:27.811047+0000","last_unstale":"2024-09-15T00:11:39.876331+0000","last_undegraded":"2024-09-15T00:11:39.876331+0000","last_fullsized":"2024-09-15T00:11:39.876331+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1089,"log_dups_size":0,"ondisk_log_size":1089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:34.938354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021428929999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8751,"num_read_kb":31066,"num_write":1650,"num_write_kb":11591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'472","reported_seq":5184,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461560+0000","last_change":"2024-09-15T00:11:16.821809+0000","last_active":"2024-09-15T00:11:40.461560+0000","last_peered":"2024-09-15T00:11:40.461560+0000","last_clean":"2024-09-15T00:11:40.461560+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:11:40.461560+0000","last_undegraded":"2024-09-15T00:11:40.461560+0000","last_fullsized":"2024-09-15T00:11:40.461560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:55:03.400561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012186440000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6281,"num_read_kb":25421,"num_write":438,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"137'321","reported_seq":4493,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876475+0000","last_change":"2024-09-15T00:11:16.818687+0000","last_active":"2024-09-15T00:11:39.876475+0000","last_peered":"2024-09-15T00:11:39.876475+0000","last_clean":"2024-09-15T00:11:39.876475+0000","last_became_active":"2024-09-15T00:10:58.347480+0000","last_became_peered":"2024-09-15T00:10:58.347480+0000","last_unstale":"2024-09-15T00:11:39.876475+0000","last_undegraded":"2024-09-15T00:11:39.876475+0000","last_fullsized":"2024-09-15T00:11:39.876475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:57.538275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015201839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3806,"num_read_kb":22330,"num_write":293,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"135'777","reported_seq":5222,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876408+0000","last_change":"2024-09-15T00:11:16.819308+0000","last_active":"2024-09-15T00:11:39.876408+0000","last_peered":"2024-09-15T00:11:39.876408+0000","last_clean":"2024-09-15T00:11:39.876408+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:11:39.876408+0000","last_undegraded":"2024-09-15T00:11:39.876408+0000","last_fullsized":"2024-09-15T00:11:39.876408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:38.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018611719999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7063,"num_read_kb":22283,"num_write":819,"num_write_kb":6499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"136'345","reported_seq":6247,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876066+0000","last_change":"2024-09-15T00:11:26.810552+0000","last_active":"2024-09-15T00:11:39.876066+0000","last_peered":"2024-09-15T00:11:39.876066+0000","last_clean":"2024-09-15T00:11:39.876066+0000","last_became_active":"2024-09-15T00:11:26.809733+0000","last_became_peered":"2024-09-15T00:11:26.809733+0000","last_unstale":"2024-09-15T00:11:39.876066+0000","last_undegraded":"2024-09-15T00:11:39.876066+0000","last_fullsized":"2024-09-15T00:11:39.876066+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:03.405500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014647,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5082,"num_read_kb":32663,"num_write":383,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"134'1204","reported_seq":4832,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461699+0000","last_change":"2024-09-15T00:11:16.821784+0000","last_active":"2024-09-15T00:11:40.461699+0000","last_peered":"2024-09-15T00:11:40.461699+0000","last_clean":"2024-09-15T00:11:40.461699+0000","last_became_active":"2024-09-15T00:10:58.354658+0000","last_became_peered":"2024-09-15T00:10:58.354658+0000","last_unstale":"2024-09-15T00:11:40.461699+0000","last_undegraded":"2024-09-15T00:11:40.461699+0000","last_fullsized":"2024-09-15T00:11:40.461699+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:36.660704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185029,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7974,"num_read_kb":21277,"num_write":1815,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'1278","reported_seq":10322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875220+0000","last_change":"2024-09-15T00:11:27.815741+0000","last_active":"2024-09-15T00:11:39.875220+0000","last_peered":"2024-09-15T00:11:39.875220+0000","last_clean":"2024-09-15T00:11:39.875220+0000","last_became_active":"2024-09-15T00:11:27.813284+0000","last_became_peered":"2024-09-15T00:11:27.813284+0000","last_unstale":"2024-09-15T00:11:39.875220+0000","last_undegraded":"2024-09-15T00:11:39.875220+0000","last_fullsized":"2024-09-15T00:11:39.875220+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:58:44.187713+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010772937999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":12900,"num_read_kb":112467,"num_write":1928,"num_write_kb":43546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"137'243","reported_seq":2506,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461549+0000","last_change":"2024-09-15T00:11:16.822816+0000","last_active":"2024-09-15T00:11:40.461549+0000","last_peered":"2024-09-15T00:11:40.461549+0000","last_clean":"2024-09-15T00:11:40.461549+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:11:40.461549+0000","last_undegraded":"2024-09-15T00:11:40.461549+0000","last_fullsized":"2024-09-15T00:11:40.461549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:36:47.914784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013592509999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1867,"num_read_kb":15504,"num_write":229,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":79,"num_read":254890,"num_read_kb":1398144,"num_write":21738,"num_write_kb":428109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2462,"num_bytes_recovered":92274536,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26303,"internal_metadata":0},"log_size":19673,"ondisk_log_size":19673,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":27652,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59444,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312396,"statfs":{"total":96636764160,"available":96575893504,"internally_reserved":0,"allocated":1933312,"data_stored":811415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":118,"seq":506806140939,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34284,"kb_used_data":1876,"kb_used_omap":31,"kb_used_meta":32352,"kb_avail":94337556,"statfs":{"total":96636764160,"available":96601657344,"internally_reserved":0,"allocated":1921024,"data_stored":806387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31754,"internal_metadata":33129462},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81704,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290136,"statfs":{"total":96636764160,"available":96553099264,"internally_reserved":0,"allocated":1986560,"data_stored":1401764,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640296,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":1929216,"data_stored":811396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":811434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":133,"seq":571230650376,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39580,"kb_used_data":1924,"kb_used_omap":28,"kb_used_meta":37603,"kb_avail":94332260,"statfs":{"total":96636764160,"available":96596234240,"internally_reserved":0,"allocated":1970176,"data_stored":1401764,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28896,"internal_metadata":38506272},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15263,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11040,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:43.342 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T00:11:43.343 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-15T00:11:43.653 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T00:11:43.667 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T00:11:43.667 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:43.956 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:43.956 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:11:43.968 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T00:11:42.883567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":79,"num_read":255006,"num_read_kb":1398244,"num_write":21899,"num_write_kb":430527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2466,"num_bytes_recovered":93455272,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19793,"ondisk_log_size":19793,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":311692,"kb_used_data":11404,"kb_used_omap":65,"kb_used_meta":300094,"kb_avail":565919348,"statfs":{"total":579820584960,"available":579501412352,"internally_reserved":0,"allocated":11677696,"data_stored":6044160,"data_compressed":18090,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":67028,"internal_metadata":307296812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001328"},"pg_stats":[{"pgid":"3.1f","version":"138'535","reported_seq":11587,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875251+0000","last_change":"2024-09-15T00:11:28.816240+0000","last_active":"2024-09-15T00:11:39.875251+0000","last_peered":"2024-09-15T00:11:39.875251+0000","last_clean":"2024-09-15T00:11:39.875251+0000","last_became_active":"2024-09-15T00:11:28.814325+0000","last_became_peered":"2024-09-15T00:11:28.814325+0000","last_unstale":"2024-09-15T00:11:39.875251+0000","last_undegraded":"2024-09-15T00:11:39.875251+0000","last_fullsized":"2024-09-15T00:11:39.875251+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:19.249372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001206129,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12081,"num_read_kb":64593,"num_write":535,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"135'496","reported_seq":5531,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875628+0000","last_change":"2024-09-15T00:11:16.808902+0000","last_active":"2024-09-15T00:11:39.875628+0000","last_peered":"2024-09-15T00:11:39.875628+0000","last_clean":"2024-09-15T00:11:39.875628+0000","last_became_active":"2024-09-15T00:10:58.347628+0000","last_became_peered":"2024-09-15T00:10:58.347628+0000","last_unstale":"2024-09-15T00:11:39.875628+0000","last_undegraded":"2024-09-15T00:11:39.875628+0000","last_fullsized":"2024-09-15T00:11:39.875628+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:19.680950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014193910000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6067,"num_read_kb":40333,"num_write":376,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"137'826","reported_seq":5711,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876271+0000","last_change":"2024-09-15T00:11:16.819173+0000","last_active":"2024-09-15T00:11:39.876271+0000","last_peered":"2024-09-15T00:11:39.876271+0000","last_clean":"2024-09-15T00:11:39.876271+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:11:39.876271+0000","last_undegraded":"2024-09-15T00:11:39.876271+0000","last_fullsized":"2024-09-15T00:11:39.876271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:38.089567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019789569999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8748,"num_read_kb":37428,"num_write":973,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"138'512","reported_seq":6131,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876263+0000","last_change":"2024-09-15T00:11:16.818860+0000","last_active":"2024-09-15T00:11:39.876263+0000","last_peered":"2024-09-15T00:11:39.876263+0000","last_clean":"2024-09-15T00:11:39.876263+0000","last_became_active":"2024-09-15T00:10:58.347433+0000","last_became_peered":"2024-09-15T00:10:58.347433+0000","last_unstale":"2024-09-15T00:11:39.876263+0000","last_undegraded":"2024-09-15T00:11:39.876263+0000","last_fullsized":"2024-09-15T00:11:39.876263+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:34.872358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878240000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6371,"num_read_kb":39648,"num_write":614,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"137'519","reported_seq":10491,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875469+0000","last_change":"2024-09-15T00:11:27.816441+0000","last_active":"2024-09-15T00:11:39.875469+0000","last_peered":"2024-09-15T00:11:39.875469+0000","last_clean":"2024-09-15T00:11:39.875469+0000","last_became_active":"2024-09-15T00:11:27.814209+0000","last_became_peered":"2024-09-15T00:11:27.814209+0000","last_unstale":"2024-09-15T00:11:39.875469+0000","last_undegraded":"2024-09-15T00:11:39.875469+0000","last_fullsized":"2024-09-15T00:11:39.875469+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:42.793496+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016246954000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11088,"num_read_kb":77879,"num_write":574,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"137'534","reported_seq":11124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797219+0000","last_change":"2024-09-15T00:11:16.807504+0000","last_active":"2024-09-15T00:11:40.797219+0000","last_peered":"2024-09-15T00:11:40.797219+0000","last_clean":"2024-09-15T00:11:40.797219+0000","last_became_active":"2024-09-15T00:10:58.346770+0000","last_became_peered":"2024-09-15T00:10:58.346770+0000","last_unstale":"2024-09-15T00:11:40.797219+0000","last_undegraded":"2024-09-15T00:11:40.797219+0000","last_fullsized":"2024-09-15T00:11:40.797219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:59.341684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015136839999999999,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":13910,"num_read_kb":40588,"num_write":444,"num_write_kb":8754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'709","reported_seq":10797,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457803+0000","last_change":"2024-09-15T00:11:16.811299+0000","last_active":"2024-09-15T00:11:40.457803+0000","last_peered":"2024-09-15T00:11:40.457803+0000","last_clean":"2024-09-15T00:11:40.457803+0000","last_became_active":"2024-09-15T00:10:58.350007+0000","last_became_peered":"2024-09-15T00:10:58.350007+0000","last_unstale":"2024-09-15T00:11:40.457803+0000","last_undegraded":"2024-09-15T00:11:40.457803+0000","last_fullsized":"2024-09-15T00:11:40.457803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:11.344098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019477310000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13597,"num_read_kb":55399,"num_write":535,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"137'655","reported_seq":8927,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875371+0000","last_change":"2024-09-15T00:11:28.815566+0000","last_active":"2024-09-15T00:11:39.875371+0000","last_peered":"2024-09-15T00:11:39.875371+0000","last_clean":"2024-09-15T00:11:39.875371+0000","last_became_active":"2024-09-15T00:11:28.813725+0000","last_became_peered":"2024-09-15T00:11:28.813725+0000","last_unstale":"2024-09-15T00:11:39.875371+0000","last_undegraded":"2024-09-15T00:11:39.875371+0000","last_fullsized":"2024-09-15T00:11:39.875371+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:28.982527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012280349999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9415,"num_read_kb":64591,"num_write":699,"num_write_kb":14239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"136'859","reported_seq":15047,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875360+0000","last_change":"2024-09-15T00:11:27.813449+0000","last_active":"2024-09-15T00:11:39.875360+0000","last_peered":"2024-09-15T00:11:39.875360+0000","last_clean":"2024-09-15T00:11:39.875360+0000","last_became_active":"2024-09-15T00:11:27.810983+0000","last_became_peered":"2024-09-15T00:11:27.810983+0000","last_unstale":"2024-09-15T00:11:39.875360+0000","last_undegraded":"2024-09-15T00:11:39.875360+0000","last_fullsized":"2024-09-15T00:11:39.875360+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:14.048930+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050753860000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":16900,"num_read_kb":72084,"num_write":902,"num_write_kb":44641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'832","reported_seq":8179,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876262+0000","last_change":"2024-09-15T00:11:16.818288+0000","last_active":"2024-09-15T00:11:39.876262+0000","last_peered":"2024-09-15T00:11:39.876262+0000","last_clean":"2024-09-15T00:11:39.876262+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:11:39.876262+0000","last_undegraded":"2024-09-15T00:11:39.876262+0000","last_fullsized":"2024-09-15T00:11:39.876262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.421381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098720300000000008,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10216,"num_read_kb":33960,"num_write":980,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"136'460","reported_seq":7066,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461827+0000","last_change":"2024-09-15T00:11:27.811672+0000","last_active":"2024-09-15T00:11:40.461827+0000","last_peered":"2024-09-15T00:11:40.461827+0000","last_clean":"2024-09-15T00:11:40.461827+0000","last_became_active":"2024-09-15T00:11:27.810534+0000","last_became_peered":"2024-09-15T00:11:27.810534+0000","last_unstale":"2024-09-15T00:11:40.461827+0000","last_undegraded":"2024-09-15T00:11:40.461827+0000","last_fullsized":"2024-09-15T00:11:40.461827+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:22:56.578888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081705799999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6309,"num_read_kb":62113,"num_write":514,"num_write_kb":29400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"136'577","reported_seq":8459,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461730+0000","last_change":"2024-09-15T00:11:16.822160+0000","last_active":"2024-09-15T00:11:40.461730+0000","last_peered":"2024-09-15T00:11:40.461730+0000","last_clean":"2024-09-15T00:11:40.461730+0000","last_became_active":"2024-09-15T00:10:58.353801+0000","last_became_peered":"2024-09-15T00:10:58.353801+0000","last_unstale":"2024-09-15T00:11:40.461730+0000","last_undegraded":"2024-09-15T00:11:40.461730+0000","last_fullsized":"2024-09-15T00:11:40.461730+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:56:52.432800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503938,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9435,"num_read_kb":45249,"num_write":397,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":310,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461963+0000","last_change":"2024-09-15T00:10:58.352604+0000","last_active":"2024-09-15T00:11:40.461963+0000","last_peered":"2024-09-15T00:11:40.461963+0000","last_clean":"2024-09-15T00:11:40.461963+0000","last_became_active":"2024-09-15T00:10:58.352314+0000","last_became_peered":"2024-09-15T00:10:58.352314+0000","last_unstale":"2024-09-15T00:11:40.461963+0000","last_undegraded":"2024-09-15T00:11:40.461963+0000","last_fullsized":"2024-09-15T00:11:40.461963+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:11.117426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"138'455","reported_seq":3568,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875353+0000","last_change":"2024-09-15T00:11:26.812244+0000","last_active":"2024-09-15T00:11:39.875353+0000","last_peered":"2024-09-15T00:11:39.875353+0000","last_clean":"2024-09-15T00:11:39.875353+0000","last_became_active":"2024-09-15T00:11:26.810222+0000","last_became_peered":"2024-09-15T00:11:26.810222+0000","last_unstale":"2024-09-15T00:11:39.875353+0000","last_undegraded":"2024-09-15T00:11:39.875353+0000","last_fullsized":"2024-09-15T00:11:39.875353+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:04:31.939139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099603400000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2834,"num_read_kb":21154,"num_write":404,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":302,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458045+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:11:40.458045+0000","last_peered":"2024-09-15T00:11:40.458045+0000","last_clean":"2024-09-15T00:11:40.458045+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:11:40.458045+0000","last_undegraded":"2024-09-15T00:11:40.458045+0000","last_fullsized":"2024-09-15T00:11:40.458045+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'301","reported_seq":3604,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808663+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:52.093302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3137,"num_read_kb":36813,"num_write":322,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461876+0000","last_change":"2024-09-15T00:11:23.821084+0000","last_active":"2024-09-15T00:11:40.461876+0000","last_peered":"2024-09-15T00:11:40.461876+0000","last_clean":"2024-09-15T00:11:40.461876+0000","last_became_active":"2024-09-15T00:11:23.820814+0000","last_became_peered":"2024-09-15T00:11:23.820814+0000","last_unstale":"2024-09-15T00:11:40.461876+0000","last_undegraded":"2024-09-15T00:11:40.461876+0000","last_fullsized":"2024-09-15T00:11:40.461876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:05:01.106813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"138'804","reported_seq":7267,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457816+0000","last_change":"2024-09-15T00:11:16.810988+0000","last_active":"2024-09-15T00:11:40.457816+0000","last_peered":"2024-09-15T00:11:40.457816+0000","last_clean":"2024-09-15T00:11:40.457816+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:11:40.457816+0000","last_undegraded":"2024-09-15T00:11:40.457816+0000","last_fullsized":"2024-09-15T00:11:40.457816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:19:00.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015707939999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9943,"num_read_kb":67757,"num_write":840,"num_write_kb":50219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":321,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457701+0000","last_change":"2024-09-15T00:11:23.823985+0000","last_active":"2024-09-15T00:11:40.457701+0000","last_peered":"2024-09-15T00:11:40.457701+0000","last_clean":"2024-09-15T00:11:40.457701+0000","last_became_active":"2024-09-15T00:11:23.823594+0000","last_became_peered":"2024-09-15T00:11:23.823594+0000","last_unstale":"2024-09-15T00:11:40.457701+0000","last_undegraded":"2024-09-15T00:11:40.457701+0000","last_fullsized":"2024-09-15T00:11:40.457701+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:01:04.517222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'598","reported_seq":5986,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876312+0000","last_change":"2024-09-15T00:11:16.819731+0000","last_active":"2024-09-15T00:11:39.876312+0000","last_peered":"2024-09-15T00:11:39.876312+0000","last_clean":"2024-09-15T00:11:39.876312+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:11:39.876312+0000","last_undegraded":"2024-09-15T00:11:39.876312+0000","last_fullsized":"2024-09-15T00:11:39.876312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:56.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022079199999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6563,"num_read_kb":34828,"num_write":657,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"138'669","reported_seq":6107,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797327+0000","last_change":"2024-09-15T00:11:26.813300+0000","last_active":"2024-09-15T00:11:40.797327+0000","last_peered":"2024-09-15T00:11:40.797327+0000","last_clean":"2024-09-15T00:11:40.797327+0000","last_became_active":"2024-09-15T00:11:26.812061+0000","last_became_peered":"2024-09-15T00:11:26.812061+0000","last_unstale":"2024-09-15T00:11:40.797327+0000","last_undegraded":"2024-09-15T00:11:40.797327+0000","last_fullsized":"2024-09-15T00:11:40.797327+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:59:00.972440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013113269999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7232,"num_read_kb":26564,"num_write":565,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876199+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:11:39.876199+0000","last_peered":"2024-09-15T00:11:39.876199+0000","last_clean":"2024-09-15T00:11:39.876199+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:11:39.876199+0000","last_undegraded":"2024-09-15T00:11:39.876199+0000","last_fullsized":"2024-09-15T00:11:39.876199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"136'714","reported_seq":7159,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457702+0000","last_change":"2024-09-15T00:11:16.811546+0000","last_active":"2024-09-15T00:11:40.457702+0000","last_peered":"2024-09-15T00:11:40.457702+0000","last_clean":"2024-09-15T00:11:40.457702+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:11:40.457702+0000","last_undegraded":"2024-09-15T00:11:40.457702+0000","last_fullsized":"2024-09-15T00:11:40.457702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:46.467190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570948,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7803,"num_read_kb":54109,"num_write":924,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875605+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:11:39.875605+0000","last_peered":"2024-09-15T00:11:39.875605+0000","last_clean":"2024-09-15T00:11:39.875605+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:11:39.875605+0000","last_undegraded":"2024-09-15T00:11:39.875605+0000","last_fullsized":"2024-09-15T00:11:39.875605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'627","reported_seq":3714,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875090+0000","last_change":"2024-09-15T00:11:26.815022+0000","last_active":"2024-09-15T00:11:39.875090+0000","last_peered":"2024-09-15T00:11:39.875090+0000","last_clean":"2024-09-15T00:11:39.875090+0000","last_became_active":"2024-09-15T00:11:26.813953+0000","last_became_peered":"2024-09-15T00:11:26.813953+0000","last_unstale":"2024-09-15T00:11:39.875090+0000","last_undegraded":"2024-09-15T00:11:39.875090+0000","last_fullsized":"2024-09-15T00:11:39.875090+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:43:46.140096+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00071073200000000005,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5290,"num_read_kb":27132,"num_write":878,"num_write_kb":8037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2365199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":303,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461630+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:11:40.461630+0000","last_peered":"2024-09-15T00:11:40.461630+0000","last_clean":"2024-09-15T00:11:40.461630+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:11:40.461630+0000","last_undegraded":"2024-09-15T00:11:40.461630+0000","last_fullsized":"2024-09-15T00:11:40.461630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'493","reported_seq":6880,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458109+0000","last_change":"2024-09-15T00:11:16.810764+0000","last_active":"2024-09-15T00:11:40.458109+0000","last_peered":"2024-09-15T00:11:40.458109+0000","last_clean":"2024-09-15T00:11:40.458109+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:11:40.458109+0000","last_undegraded":"2024-09-15T00:11:40.458109+0000","last_fullsized":"2024-09-15T00:11:40.458109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:10:05.221091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215973,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6232,"num_read_kb":25476,"num_write":407,"num_write_kb":5284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":305,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876278+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:11:39.876278+0000","last_peered":"2024-09-15T00:11:39.876278+0000","last_clean":"2024-09-15T00:11:39.876278+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:11:39.876278+0000","last_undegraded":"2024-09-15T00:11:39.876278+0000","last_fullsized":"2024-09-15T00:11:39.876278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":482,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461680+0000","last_change":"2024-09-15T00:11:26.809502+0000","last_active":"2024-09-15T00:11:40.461680+0000","last_peered":"2024-09-15T00:11:40.461680+0000","last_clean":"2024-09-15T00:11:40.461680+0000","last_became_active":"2024-09-15T00:11:26.809081+0000","last_became_peered":"2024-09-15T00:11:26.809081+0000","last_unstale":"2024-09-15T00:11:40.461680+0000","last_undegraded":"2024-09-15T00:11:40.461680+0000","last_fullsized":"2024-09-15T00:11:40.461680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:33.161918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"136'570","reported_seq":11116,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808250+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:18.362309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098862600000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":12740,"num_read_kb":88117,"num_write":607,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"137'511","reported_seq":9033,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797237+0000","last_change":"2024-09-15T00:11:16.806418+0000","last_active":"2024-09-15T00:11:40.797237+0000","last_peered":"2024-09-15T00:11:40.797237+0000","last_clean":"2024-09-15T00:11:40.797237+0000","last_became_active":"2024-09-15T00:10:58.346994+0000","last_became_peered":"2024-09-15T00:10:58.346994+0000","last_unstale":"2024-09-15T00:11:40.797237+0000","last_undegraded":"2024-09-15T00:11:40.797237+0000","last_fullsized":"2024-09-15T00:11:40.797237+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:09:39.692848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088599499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9287,"num_read_kb":49680,"num_write":419,"num_write_kb":13143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"137'346","reported_seq":2534,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461768+0000","last_change":"2024-09-15T00:11:16.822630+0000","last_active":"2024-09-15T00:11:40.461768+0000","last_peered":"2024-09-15T00:11:40.461768+0000","last_clean":"2024-09-15T00:11:40.461768+0000","last_became_active":"2024-09-15T00:10:58.353160+0000","last_became_peered":"2024-09-15T00:10:58.353160+0000","last_unstale":"2024-09-15T00:11:40.461768+0000","last_undegraded":"2024-09-15T00:11:40.461768+0000","last_fullsized":"2024-09-15T00:11:40.461768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:32.587056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001573252,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2498,"num_read_kb":16284,"num_write":316,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"137'342","reported_seq":4084,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457821+0000","last_change":"2024-09-15T00:11:27.813271+0000","last_active":"2024-09-15T00:11:40.457821+0000","last_peered":"2024-09-15T00:11:40.457821+0000","last_clean":"2024-09-15T00:11:40.457821+0000","last_became_active":"2024-09-15T00:11:27.812092+0000","last_became_peered":"2024-09-15T00:11:27.812092+0000","last_unstale":"2024-09-15T00:11:40.457821+0000","last_undegraded":"2024-09-15T00:11:40.457821+0000","last_fullsized":"2024-09-15T00:11:40.457821+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:10:35.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000721333,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3470,"num_read_kb":33354,"num_write":301,"num_write_kb":6941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"138'1089","reported_seq":6155,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876331+0000","last_change":"2024-09-15T00:11:27.812382+0000","last_active":"2024-09-15T00:11:39.876331+0000","last_peered":"2024-09-15T00:11:39.876331+0000","last_clean":"2024-09-15T00:11:39.876331+0000","last_became_active":"2024-09-15T00:11:27.811047+0000","last_became_peered":"2024-09-15T00:11:27.811047+0000","last_unstale":"2024-09-15T00:11:39.876331+0000","last_undegraded":"2024-09-15T00:11:39.876331+0000","last_fullsized":"2024-09-15T00:11:39.876331+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1089,"log_dups_size":0,"ondisk_log_size":1089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:34.938354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021428929999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8751,"num_read_kb":31066,"num_write":1650,"num_write_kb":11591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'472","reported_seq":5184,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461560+0000","last_change":"2024-09-15T00:11:16.821809+0000","last_active":"2024-09-15T00:11:40.461560+0000","last_peered":"2024-09-15T00:11:40.461560+0000","last_clean":"2024-09-15T00:11:40.461560+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:11:40.461560+0000","last_undegraded":"2024-09-15T00:11:40.461560+0000","last_fullsized":"2024-09-15T00:11:40.461560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:55:03.400561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012186440000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6281,"num_read_kb":25421,"num_write":438,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"137'321","reported_seq":4493,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876475+0000","last_change":"2024-09-15T00:11:16.818687+0000","last_active":"2024-09-15T00:11:39.876475+0000","last_peered":"2024-09-15T00:11:39.876475+0000","last_clean":"2024-09-15T00:11:39.876475+0000","last_became_active":"2024-09-15T00:10:58.347480+0000","last_became_peered":"2024-09-15T00:10:58.347480+0000","last_unstale":"2024-09-15T00:11:39.876475+0000","last_undegraded":"2024-09-15T00:11:39.876475+0000","last_fullsized":"2024-09-15T00:11:39.876475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:57.538275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015201839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3806,"num_read_kb":22330,"num_write":293,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"135'777","reported_seq":5222,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876408+0000","last_change":"2024-09-15T00:11:16.819308+0000","last_active":"2024-09-15T00:11:39.876408+0000","last_peered":"2024-09-15T00:11:39.876408+0000","last_clean":"2024-09-15T00:11:39.876408+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:11:39.876408+0000","last_undegraded":"2024-09-15T00:11:39.876408+0000","last_fullsized":"2024-09-15T00:11:39.876408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:38.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018611719999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7063,"num_read_kb":22283,"num_write":819,"num_write_kb":6499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"136'345","reported_seq":6247,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876066+0000","last_change":"2024-09-15T00:11:26.810552+0000","last_active":"2024-09-15T00:11:39.876066+0000","last_peered":"2024-09-15T00:11:39.876066+0000","last_clean":"2024-09-15T00:11:39.876066+0000","last_became_active":"2024-09-15T00:11:26.809733+0000","last_became_peered":"2024-09-15T00:11:26.809733+0000","last_unstale":"2024-09-15T00:11:39.876066+0000","last_undegraded":"2024-09-15T00:11:39.876066+0000","last_fullsized":"2024-09-15T00:11:39.876066+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:03.405500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014647,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5082,"num_read_kb":32663,"num_write":383,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"134'1204","reported_seq":4832,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461699+0000","last_change":"2024-09-15T00:11:16.821784+0000","last_active":"2024-09-15T00:11:40.461699+0000","last_peered":"2024-09-15T00:11:40.461699+0000","last_clean":"2024-09-15T00:11:40.461699+0000","last_became_active":"2024-09-15T00:10:58.354658+0000","last_became_peered":"2024-09-15T00:10:58.354658+0000","last_unstale":"2024-09-15T00:11:40.461699+0000","last_undegraded":"2024-09-15T00:11:40.461699+0000","last_fullsized":"2024-09-15T00:11:40.461699+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:36.660704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185029,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7974,"num_read_kb":21277,"num_write":1815,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'1278","reported_seq":10322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875220+0000","last_change":"2024-09-15T00:11:27.815741+0000","last_active":"2024-09-15T00:11:39.875220+0000","last_peered":"2024-09-15T00:11:39.875220+0000","last_clean":"2024-09-15T00:11:39.875220+0000","last_became_active":"2024-09-15T00:11:27.813284+0000","last_became_peered":"2024-09-15T00:11:27.813284+0000","last_unstale":"2024-09-15T00:11:39.875220+0000","last_undegraded":"2024-09-15T00:11:39.875220+0000","last_fullsized":"2024-09-15T00:11:39.875220+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:58:44.187713+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010772937999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":12900,"num_read_kb":112467,"num_write":1928,"num_write_kb":43546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"137'243","reported_seq":2506,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461549+0000","last_change":"2024-09-15T00:11:16.822816+0000","last_active":"2024-09-15T00:11:40.461549+0000","last_peered":"2024-09-15T00:11:40.461549+0000","last_clean":"2024-09-15T00:11:40.461549+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:11:40.461549+0000","last_undegraded":"2024-09-15T00:11:40.461549+0000","last_fullsized":"2024-09-15T00:11:40.461549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:36:47.914784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013592509999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1867,"num_read_kb":15504,"num_write":229,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":79,"num_read":254890,"num_read_kb":1398144,"num_write":21738,"num_write_kb":428109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2462,"num_bytes_recovered":92274536,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26303,"internal_metadata":0},"log_size":19673,"ondisk_log_size":19673,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":27652,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59444,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312396,"statfs":{"total":96636764160,"available":96575893504,"internally_reserved":0,"allocated":1933312,"data_stored":811415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":118,"seq":506806140939,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34284,"kb_used_data":1876,"kb_used_omap":31,"kb_used_meta":32352,"kb_avail":94337556,"statfs":{"total":96636764160,"available":96601657344,"internally_reserved":0,"allocated":1921024,"data_stored":806387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31754,"internal_metadata":33129462},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81704,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290136,"statfs":{"total":96636764160,"available":96553099264,"internally_reserved":0,"allocated":1986560,"data_stored":1401764,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640296,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":1929216,"data_stored":811396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":811434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":133,"seq":571230650376,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39580,"kb_used_data":1924,"kb_used_omap":28,"kb_used_meta":37603,"kb_avail":94332260,"statfs":{"total":96636764160,"available":96596234240,"internally_reserved":0,"allocated":1970176,"data_stored":1401764,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28896,"internal_metadata":38506272},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15263,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11040,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:43.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:44.258 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:44.259 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:11:44.270 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T00:11:42.883567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":79,"num_read":255006,"num_read_kb":1398244,"num_write":21899,"num_write_kb":430527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2466,"num_bytes_recovered":93455272,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19793,"ondisk_log_size":19793,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":311692,"kb_used_data":11404,"kb_used_omap":65,"kb_used_meta":300094,"kb_avail":565919348,"statfs":{"total":579820584960,"available":579501412352,"internally_reserved":0,"allocated":11677696,"data_stored":6044160,"data_compressed":18090,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":67028,"internal_metadata":307296812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001328"},"pg_stats":[{"pgid":"3.1f","version":"138'535","reported_seq":11587,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875251+0000","last_change":"2024-09-15T00:11:28.816240+0000","last_active":"2024-09-15T00:11:39.875251+0000","last_peered":"2024-09-15T00:11:39.875251+0000","last_clean":"2024-09-15T00:11:39.875251+0000","last_became_active":"2024-09-15T00:11:28.814325+0000","last_became_peered":"2024-09-15T00:11:28.814325+0000","last_unstale":"2024-09-15T00:11:39.875251+0000","last_undegraded":"2024-09-15T00:11:39.875251+0000","last_fullsized":"2024-09-15T00:11:39.875251+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:19.249372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001206129,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12081,"num_read_kb":64593,"num_write":535,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"135'496","reported_seq":5531,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875628+0000","last_change":"2024-09-15T00:11:16.808902+0000","last_active":"2024-09-15T00:11:39.875628+0000","last_peered":"2024-09-15T00:11:39.875628+0000","last_clean":"2024-09-15T00:11:39.875628+0000","last_became_active":"2024-09-15T00:10:58.347628+0000","last_became_peered":"2024-09-15T00:10:58.347628+0000","last_unstale":"2024-09-15T00:11:39.875628+0000","last_undegraded":"2024-09-15T00:11:39.875628+0000","last_fullsized":"2024-09-15T00:11:39.875628+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:19.680950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014193910000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6067,"num_read_kb":40333,"num_write":376,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"137'826","reported_seq":5711,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876271+0000","last_change":"2024-09-15T00:11:16.819173+0000","last_active":"2024-09-15T00:11:39.876271+0000","last_peered":"2024-09-15T00:11:39.876271+0000","last_clean":"2024-09-15T00:11:39.876271+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:11:39.876271+0000","last_undegraded":"2024-09-15T00:11:39.876271+0000","last_fullsized":"2024-09-15T00:11:39.876271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:38.089567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019789569999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8748,"num_read_kb":37428,"num_write":973,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"138'512","reported_seq":6131,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876263+0000","last_change":"2024-09-15T00:11:16.818860+0000","last_active":"2024-09-15T00:11:39.876263+0000","last_peered":"2024-09-15T00:11:39.876263+0000","last_clean":"2024-09-15T00:11:39.876263+0000","last_became_active":"2024-09-15T00:10:58.347433+0000","last_became_peered":"2024-09-15T00:10:58.347433+0000","last_unstale":"2024-09-15T00:11:39.876263+0000","last_undegraded":"2024-09-15T00:11:39.876263+0000","last_fullsized":"2024-09-15T00:11:39.876263+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:34.872358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878240000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6371,"num_read_kb":39648,"num_write":614,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"137'519","reported_seq":10491,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875469+0000","last_change":"2024-09-15T00:11:27.816441+0000","last_active":"2024-09-15T00:11:39.875469+0000","last_peered":"2024-09-15T00:11:39.875469+0000","last_clean":"2024-09-15T00:11:39.875469+0000","last_became_active":"2024-09-15T00:11:27.814209+0000","last_became_peered":"2024-09-15T00:11:27.814209+0000","last_unstale":"2024-09-15T00:11:39.875469+0000","last_undegraded":"2024-09-15T00:11:39.875469+0000","last_fullsized":"2024-09-15T00:11:39.875469+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:42.793496+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016246954000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11088,"num_read_kb":77879,"num_write":574,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"137'534","reported_seq":11124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797219+0000","last_change":"2024-09-15T00:11:16.807504+0000","last_active":"2024-09-15T00:11:40.797219+0000","last_peered":"2024-09-15T00:11:40.797219+0000","last_clean":"2024-09-15T00:11:40.797219+0000","last_became_active":"2024-09-15T00:10:58.346770+0000","last_became_peered":"2024-09-15T00:10:58.346770+0000","last_unstale":"2024-09-15T00:11:40.797219+0000","last_undegraded":"2024-09-15T00:11:40.797219+0000","last_fullsized":"2024-09-15T00:11:40.797219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:59.341684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015136839999999999,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":13910,"num_read_kb":40588,"num_write":444,"num_write_kb":8754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'709","reported_seq":10797,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457803+0000","last_change":"2024-09-15T00:11:16.811299+0000","last_active":"2024-09-15T00:11:40.457803+0000","last_peered":"2024-09-15T00:11:40.457803+0000","last_clean":"2024-09-15T00:11:40.457803+0000","last_became_active":"2024-09-15T00:10:58.350007+0000","last_became_peered":"2024-09-15T00:10:58.350007+0000","last_unstale":"2024-09-15T00:11:40.457803+0000","last_undegraded":"2024-09-15T00:11:40.457803+0000","last_fullsized":"2024-09-15T00:11:40.457803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:11.344098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019477310000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13597,"num_read_kb":55399,"num_write":535,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"137'655","reported_seq":8927,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875371+0000","last_change":"2024-09-15T00:11:28.815566+0000","last_active":"2024-09-15T00:11:39.875371+0000","last_peered":"2024-09-15T00:11:39.875371+0000","last_clean":"2024-09-15T00:11:39.875371+0000","last_became_active":"2024-09-15T00:11:28.813725+0000","last_became_peered":"2024-09-15T00:11:28.813725+0000","last_unstale":"2024-09-15T00:11:39.875371+0000","last_undegraded":"2024-09-15T00:11:39.875371+0000","last_fullsized":"2024-09-15T00:11:39.875371+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:28.982527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012280349999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9415,"num_read_kb":64591,"num_write":699,"num_write_kb":14239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"136'859","reported_seq":15047,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875360+0000","last_change":"2024-09-15T00:11:27.813449+0000","last_active":"2024-09-15T00:11:39.875360+0000","last_peered":"2024-09-15T00:11:39.875360+0000","last_clean":"2024-09-15T00:11:39.875360+0000","last_became_active":"2024-09-15T00:11:27.810983+0000","last_became_peered":"2024-09-15T00:11:27.810983+0000","last_unstale":"2024-09-15T00:11:39.875360+0000","last_undegraded":"2024-09-15T00:11:39.875360+0000","last_fullsized":"2024-09-15T00:11:39.875360+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:14.048930+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050753860000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":16900,"num_read_kb":72084,"num_write":902,"num_write_kb":44641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'832","reported_seq":8179,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876262+0000","last_change":"2024-09-15T00:11:16.818288+0000","last_active":"2024-09-15T00:11:39.876262+0000","last_peered":"2024-09-15T00:11:39.876262+0000","last_clean":"2024-09-15T00:11:39.876262+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:11:39.876262+0000","last_undegraded":"2024-09-15T00:11:39.876262+0000","last_fullsized":"2024-09-15T00:11:39.876262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.421381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098720300000000008,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10216,"num_read_kb":33960,"num_write":980,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"136'460","reported_seq":7066,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461827+0000","last_change":"2024-09-15T00:11:27.811672+0000","last_active":"2024-09-15T00:11:40.461827+0000","last_peered":"2024-09-15T00:11:40.461827+0000","last_clean":"2024-09-15T00:11:40.461827+0000","last_became_active":"2024-09-15T00:11:27.810534+0000","last_became_peered":"2024-09-15T00:11:27.810534+0000","last_unstale":"2024-09-15T00:11:40.461827+0000","last_undegraded":"2024-09-15T00:11:40.461827+0000","last_fullsized":"2024-09-15T00:11:40.461827+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:22:56.578888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081705799999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6309,"num_read_kb":62113,"num_write":514,"num_write_kb":29400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"136'577","reported_seq":8459,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461730+0000","last_change":"2024-09-15T00:11:16.822160+0000","last_active":"2024-09-15T00:11:40.461730+0000","last_peered":"2024-09-15T00:11:40.461730+0000","last_clean":"2024-09-15T00:11:40.461730+0000","last_became_active":"2024-09-15T00:10:58.353801+0000","last_became_peered":"2024-09-15T00:10:58.353801+0000","last_unstale":"2024-09-15T00:11:40.461730+0000","last_undegraded":"2024-09-15T00:11:40.461730+0000","last_fullsized":"2024-09-15T00:11:40.461730+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:56:52.432800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503938,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9435,"num_read_kb":45249,"num_write":397,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":310,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461963+0000","last_change":"2024-09-15T00:10:58.352604+0000","last_active":"2024-09-15T00:11:40.461963+0000","last_peered":"2024-09-15T00:11:40.461963+0000","last_clean":"2024-09-15T00:11:40.461963+0000","last_became_active":"2024-09-15T00:10:58.352314+0000","last_became_peered":"2024-09-15T00:10:58.352314+0000","last_unstale":"2024-09-15T00:11:40.461963+0000","last_undegraded":"2024-09-15T00:11:40.461963+0000","last_fullsized":"2024-09-15T00:11:40.461963+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:11.117426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"138'455","reported_seq":3568,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875353+0000","last_change":"2024-09-15T00:11:26.812244+0000","last_active":"2024-09-15T00:11:39.875353+0000","last_peered":"2024-09-15T00:11:39.875353+0000","last_clean":"2024-09-15T00:11:39.875353+0000","last_became_active":"2024-09-15T00:11:26.810222+0000","last_became_peered":"2024-09-15T00:11:26.810222+0000","last_unstale":"2024-09-15T00:11:39.875353+0000","last_undegraded":"2024-09-15T00:11:39.875353+0000","last_fullsized":"2024-09-15T00:11:39.875353+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:04:31.939139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099603400000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2834,"num_read_kb":21154,"num_write":404,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":302,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458045+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:11:40.458045+0000","last_peered":"2024-09-15T00:11:40.458045+0000","last_clean":"2024-09-15T00:11:40.458045+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:11:40.458045+0000","last_undegraded":"2024-09-15T00:11:40.458045+0000","last_fullsized":"2024-09-15T00:11:40.458045+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'301","reported_seq":3604,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808663+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:52.093302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3137,"num_read_kb":36813,"num_write":322,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461876+0000","last_change":"2024-09-15T00:11:23.821084+0000","last_active":"2024-09-15T00:11:40.461876+0000","last_peered":"2024-09-15T00:11:40.461876+0000","last_clean":"2024-09-15T00:11:40.461876+0000","last_became_active":"2024-09-15T00:11:23.820814+0000","last_became_peered":"2024-09-15T00:11:23.820814+0000","last_unstale":"2024-09-15T00:11:40.461876+0000","last_undegraded":"2024-09-15T00:11:40.461876+0000","last_fullsized":"2024-09-15T00:11:40.461876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:05:01.106813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"138'804","reported_seq":7267,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457816+0000","last_change":"2024-09-15T00:11:16.810988+0000","last_active":"2024-09-15T00:11:40.457816+0000","last_peered":"2024-09-15T00:11:40.457816+0000","last_clean":"2024-09-15T00:11:40.457816+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:11:40.457816+0000","last_undegraded":"2024-09-15T00:11:40.457816+0000","last_fullsized":"2024-09-15T00:11:40.457816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:19:00.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015707939999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9943,"num_read_kb":67757,"num_write":840,"num_write_kb":50219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":321,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457701+0000","last_change":"2024-09-15T00:11:23.823985+0000","last_active":"2024-09-15T00:11:40.457701+0000","last_peered":"2024-09-15T00:11:40.457701+0000","last_clean":"2024-09-15T00:11:40.457701+0000","last_became_active":"2024-09-15T00:11:23.823594+0000","last_became_peered":"2024-09-15T00:11:23.823594+0000","last_unstale":"2024-09-15T00:11:40.457701+0000","last_undegraded":"2024-09-15T00:11:40.457701+0000","last_fullsized":"2024-09-15T00:11:40.457701+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:01:04.517222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'598","reported_seq":5986,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876312+0000","last_change":"2024-09-15T00:11:16.819731+0000","last_active":"2024-09-15T00:11:39.876312+0000","last_peered":"2024-09-15T00:11:39.876312+0000","last_clean":"2024-09-15T00:11:39.876312+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:11:39.876312+0000","last_undegraded":"2024-09-15T00:11:39.876312+0000","last_fullsized":"2024-09-15T00:11:39.876312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:56.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022079199999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6563,"num_read_kb":34828,"num_write":657,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"138'669","reported_seq":6107,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797327+0000","last_change":"2024-09-15T00:11:26.813300+0000","last_active":"2024-09-15T00:11:40.797327+0000","last_peered":"2024-09-15T00:11:40.797327+0000","last_clean":"2024-09-15T00:11:40.797327+0000","last_became_active":"2024-09-15T00:11:26.812061+0000","last_became_peered":"2024-09-15T00:11:26.812061+0000","last_unstale":"2024-09-15T00:11:40.797327+0000","last_undegraded":"2024-09-15T00:11:40.797327+0000","last_fullsized":"2024-09-15T00:11:40.797327+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:59:00.972440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013113269999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7232,"num_read_kb":26564,"num_write":565,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876199+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:11:39.876199+0000","last_peered":"2024-09-15T00:11:39.876199+0000","last_clean":"2024-09-15T00:11:39.876199+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:11:39.876199+0000","last_undegraded":"2024-09-15T00:11:39.876199+0000","last_fullsized":"2024-09-15T00:11:39.876199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"136'714","reported_seq":7159,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457702+0000","last_change":"2024-09-15T00:11:16.811546+0000","last_active":"2024-09-15T00:11:40.457702+0000","last_peered":"2024-09-15T00:11:40.457702+0000","last_clean":"2024-09-15T00:11:40.457702+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:11:40.457702+0000","last_undegraded":"2024-09-15T00:11:40.457702+0000","last_fullsized":"2024-09-15T00:11:40.457702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:46.467190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570948,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7803,"num_read_kb":54109,"num_write":924,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875605+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:11:39.875605+0000","last_peered":"2024-09-15T00:11:39.875605+0000","last_clean":"2024-09-15T00:11:39.875605+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:11:39.875605+0000","last_undegraded":"2024-09-15T00:11:39.875605+0000","last_fullsized":"2024-09-15T00:11:39.875605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'627","reported_seq":3714,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875090+0000","last_change":"2024-09-15T00:11:26.815022+0000","last_active":"2024-09-15T00:11:39.875090+0000","last_peered":"2024-09-15T00:11:39.875090+0000","last_clean":"2024-09-15T00:11:39.875090+0000","last_became_active":"2024-09-15T00:11:26.813953+0000","last_became_peered":"2024-09-15T00:11:26.813953+0000","last_unstale":"2024-09-15T00:11:39.875090+0000","last_undegraded":"2024-09-15T00:11:39.875090+0000","last_fullsized":"2024-09-15T00:11:39.875090+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:43:46.140096+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00071073200000000005,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5290,"num_read_kb":27132,"num_write":878,"num_write_kb":8037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2365199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":303,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461630+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:11:40.461630+0000","last_peered":"2024-09-15T00:11:40.461630+0000","last_clean":"2024-09-15T00:11:40.461630+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:11:40.461630+0000","last_undegraded":"2024-09-15T00:11:40.461630+0000","last_fullsized":"2024-09-15T00:11:40.461630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'493","reported_seq":6880,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458109+0000","last_change":"2024-09-15T00:11:16.810764+0000","last_active":"2024-09-15T00:11:40.458109+0000","last_peered":"2024-09-15T00:11:40.458109+0000","last_clean":"2024-09-15T00:11:40.458109+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:11:40.458109+0000","last_undegraded":"2024-09-15T00:11:40.458109+0000","last_fullsized":"2024-09-15T00:11:40.458109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:10:05.221091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215973,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6232,"num_read_kb":25476,"num_write":407,"num_write_kb":5284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":305,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876278+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:11:39.876278+0000","last_peered":"2024-09-15T00:11:39.876278+0000","last_clean":"2024-09-15T00:11:39.876278+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:11:39.876278+0000","last_undegraded":"2024-09-15T00:11:39.876278+0000","last_fullsized":"2024-09-15T00:11:39.876278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":482,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461680+0000","last_change":"2024-09-15T00:11:26.809502+0000","last_active":"2024-09-15T00:11:40.461680+0000","last_peered":"2024-09-15T00:11:40.461680+0000","last_clean":"2024-09-15T00:11:40.461680+0000","last_became_active":"2024-09-15T00:11:26.809081+0000","last_became_peered":"2024-09-15T00:11:26.809081+0000","last_unstale":"2024-09-15T00:11:40.461680+0000","last_undegraded":"2024-09-15T00:11:40.461680+0000","last_fullsized":"2024-09-15T00:11:40.461680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:33.161918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"136'570","reported_seq":11116,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808250+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:18.362309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098862600000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":12740,"num_read_kb":88117,"num_write":607,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"137'511","reported_seq":9033,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797237+0000","last_change":"2024-09-15T00:11:16.806418+0000","last_active":"2024-09-15T00:11:40.797237+0000","last_peered":"2024-09-15T00:11:40.797237+0000","last_clean":"2024-09-15T00:11:40.797237+0000","last_became_active":"2024-09-15T00:10:58.346994+0000","last_became_peered":"2024-09-15T00:10:58.346994+0000","last_unstale":"2024-09-15T00:11:40.797237+0000","last_undegraded":"2024-09-15T00:11:40.797237+0000","last_fullsized":"2024-09-15T00:11:40.797237+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:09:39.692848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088599499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9287,"num_read_kb":49680,"num_write":419,"num_write_kb":13143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"137'346","reported_seq":2534,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461768+0000","last_change":"2024-09-15T00:11:16.822630+0000","last_active":"2024-09-15T00:11:40.461768+0000","last_peered":"2024-09-15T00:11:40.461768+0000","last_clean":"2024-09-15T00:11:40.461768+0000","last_became_active":"2024-09-15T00:10:58.353160+0000","last_became_peered":"2024-09-15T00:10:58.353160+0000","last_unstale":"2024-09-15T00:11:40.461768+0000","last_undegraded":"2024-09-15T00:11:40.461768+0000","last_fullsized":"2024-09-15T00:11:40.461768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:32.587056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001573252,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2498,"num_read_kb":16284,"num_write":316,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"137'342","reported_seq":4084,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457821+0000","last_change":"2024-09-15T00:11:27.813271+0000","last_active":"2024-09-15T00:11:40.457821+0000","last_peered":"2024-09-15T00:11:40.457821+0000","last_clean":"2024-09-15T00:11:40.457821+0000","last_became_active":"2024-09-15T00:11:27.812092+0000","last_became_peered":"2024-09-15T00:11:27.812092+0000","last_unstale":"2024-09-15T00:11:40.457821+0000","last_undegraded":"2024-09-15T00:11:40.457821+0000","last_fullsized":"2024-09-15T00:11:40.457821+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:10:35.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000721333,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3470,"num_read_kb":33354,"num_write":301,"num_write_kb":6941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"138'1089","reported_seq":6155,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876331+0000","last_change":"2024-09-15T00:11:27.812382+0000","last_active":"2024-09-15T00:11:39.876331+0000","last_peered":"2024-09-15T00:11:39.876331+0000","last_clean":"2024-09-15T00:11:39.876331+0000","last_became_active":"2024-09-15T00:11:27.811047+0000","last_became_peered":"2024-09-15T00:11:27.811047+0000","last_unstale":"2024-09-15T00:11:39.876331+0000","last_undegraded":"2024-09-15T00:11:39.876331+0000","last_fullsized":"2024-09-15T00:11:39.876331+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1089,"log_dups_size":0,"ondisk_log_size":1089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:34.938354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021428929999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8751,"num_read_kb":31066,"num_write":1650,"num_write_kb":11591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'472","reported_seq":5184,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461560+0000","last_change":"2024-09-15T00:11:16.821809+0000","last_active":"2024-09-15T00:11:40.461560+0000","last_peered":"2024-09-15T00:11:40.461560+0000","last_clean":"2024-09-15T00:11:40.461560+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:11:40.461560+0000","last_undegraded":"2024-09-15T00:11:40.461560+0000","last_fullsized":"2024-09-15T00:11:40.461560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:55:03.400561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012186440000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6281,"num_read_kb":25421,"num_write":438,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"137'321","reported_seq":4493,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876475+0000","last_change":"2024-09-15T00:11:16.818687+0000","last_active":"2024-09-15T00:11:39.876475+0000","last_peered":"2024-09-15T00:11:39.876475+0000","last_clean":"2024-09-15T00:11:39.876475+0000","last_became_active":"2024-09-15T00:10:58.347480+0000","last_became_peered":"2024-09-15T00:10:58.347480+0000","last_unstale":"2024-09-15T00:11:39.876475+0000","last_undegraded":"2024-09-15T00:11:39.876475+0000","last_fullsized":"2024-09-15T00:11:39.876475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:57.538275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015201839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3806,"num_read_kb":22330,"num_write":293,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"135'777","reported_seq":5222,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876408+0000","last_change":"2024-09-15T00:11:16.819308+0000","last_active":"2024-09-15T00:11:39.876408+0000","last_peered":"2024-09-15T00:11:39.876408+0000","last_clean":"2024-09-15T00:11:39.876408+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:11:39.876408+0000","last_undegraded":"2024-09-15T00:11:39.876408+0000","last_fullsized":"2024-09-15T00:11:39.876408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:38.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018611719999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7063,"num_read_kb":22283,"num_write":819,"num_write_kb":6499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"136'345","reported_seq":6247,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876066+0000","last_change":"2024-09-15T00:11:26.810552+0000","last_active":"2024-09-15T00:11:39.876066+0000","last_peered":"2024-09-15T00:11:39.876066+0000","last_clean":"2024-09-15T00:11:39.876066+0000","last_became_active":"2024-09-15T00:11:26.809733+0000","last_became_peered":"2024-09-15T00:11:26.809733+0000","last_unstale":"2024-09-15T00:11:39.876066+0000","last_undegraded":"2024-09-15T00:11:39.876066+0000","last_fullsized":"2024-09-15T00:11:39.876066+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:03.405500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014647,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5082,"num_read_kb":32663,"num_write":383,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"134'1204","reported_seq":4832,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461699+0000","last_change":"2024-09-15T00:11:16.821784+0000","last_active":"2024-09-15T00:11:40.461699+0000","last_peered":"2024-09-15T00:11:40.461699+0000","last_clean":"2024-09-15T00:11:40.461699+0000","last_became_active":"2024-09-15T00:10:58.354658+0000","last_became_peered":"2024-09-15T00:10:58.354658+0000","last_unstale":"2024-09-15T00:11:40.461699+0000","last_undegraded":"2024-09-15T00:11:40.461699+0000","last_fullsized":"2024-09-15T00:11:40.461699+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:36.660704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185029,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7974,"num_read_kb":21277,"num_write":1815,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'1278","reported_seq":10322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875220+0000","last_change":"2024-09-15T00:11:27.815741+0000","last_active":"2024-09-15T00:11:39.875220+0000","last_peered":"2024-09-15T00:11:39.875220+0000","last_clean":"2024-09-15T00:11:39.875220+0000","last_became_active":"2024-09-15T00:11:27.813284+0000","last_became_peered":"2024-09-15T00:11:27.813284+0000","last_unstale":"2024-09-15T00:11:39.875220+0000","last_undegraded":"2024-09-15T00:11:39.875220+0000","last_fullsized":"2024-09-15T00:11:39.875220+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:58:44.187713+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010772937999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":12900,"num_read_kb":112467,"num_write":1928,"num_write_kb":43546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"137'243","reported_seq":2506,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461549+0000","last_change":"2024-09-15T00:11:16.822816+0000","last_active":"2024-09-15T00:11:40.461549+0000","last_peered":"2024-09-15T00:11:40.461549+0000","last_clean":"2024-09-15T00:11:40.461549+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:11:40.461549+0000","last_undegraded":"2024-09-15T00:11:40.461549+0000","last_fullsized":"2024-09-15T00:11:40.461549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:36:47.914784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013592509999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1867,"num_read_kb":15504,"num_write":229,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":79,"num_read":254890,"num_read_kb":1398144,"num_write":21738,"num_write_kb":428109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2462,"num_bytes_recovered":92274536,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26303,"internal_metadata":0},"log_size":19673,"ondisk_log_size":19673,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":27652,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59444,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312396,"statfs":{"total":96636764160,"available":96575893504,"internally_reserved":0,"allocated":1933312,"data_stored":811415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":118,"seq":506806140939,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34284,"kb_used_data":1876,"kb_used_omap":31,"kb_used_meta":32352,"kb_avail":94337556,"statfs":{"total":96636764160,"available":96601657344,"internally_reserved":0,"allocated":1921024,"data_stored":806387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31754,"internal_metadata":33129462},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81704,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290136,"statfs":{"total":96636764160,"available":96553099264,"internally_reserved":0,"allocated":1986560,"data_stored":1401764,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640296,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":1929216,"data_stored":811396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":811434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":133,"seq":571230650376,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39580,"kb_used_data":1924,"kb_used_omap":28,"kb_used_meta":37603,"kb_avail":94332260,"statfs":{"total":96636764160,"available":96596234240,"internally_reserved":0,"allocated":1970176,"data_stored":1401764,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28896,"internal_metadata":38506272},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15263,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11040,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:44.272 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T00:11:44.272 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:44.564 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:11:44.575 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T00:11:42.883567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":79,"num_read":255006,"num_read_kb":1398244,"num_write":21899,"num_write_kb":430527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2466,"num_bytes_recovered":93455272,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19793,"ondisk_log_size":19793,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":311692,"kb_used_data":11404,"kb_used_omap":65,"kb_used_meta":300094,"kb_avail":565919348,"statfs":{"total":579820584960,"available":579501412352,"internally_reserved":0,"allocated":11677696,"data_stored":6044160,"data_compressed":18090,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":67028,"internal_metadata":307296812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001328"},"pg_stats":[{"pgid":"3.1f","version":"138'535","reported_seq":11587,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875251+0000","last_change":"2024-09-15T00:11:28.816240+0000","last_active":"2024-09-15T00:11:39.875251+0000","last_peered":"2024-09-15T00:11:39.875251+0000","last_clean":"2024-09-15T00:11:39.875251+0000","last_became_active":"2024-09-15T00:11:28.814325+0000","last_became_peered":"2024-09-15T00:11:28.814325+0000","last_unstale":"2024-09-15T00:11:39.875251+0000","last_undegraded":"2024-09-15T00:11:39.875251+0000","last_fullsized":"2024-09-15T00:11:39.875251+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:19.249372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001206129,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12081,"num_read_kb":64593,"num_write":535,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"135'496","reported_seq":5531,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875628+0000","last_change":"2024-09-15T00:11:16.808902+0000","last_active":"2024-09-15T00:11:39.875628+0000","last_peered":"2024-09-15T00:11:39.875628+0000","last_clean":"2024-09-15T00:11:39.875628+0000","last_became_active":"2024-09-15T00:10:58.347628+0000","last_became_peered":"2024-09-15T00:10:58.347628+0000","last_unstale":"2024-09-15T00:11:39.875628+0000","last_undegraded":"2024-09-15T00:11:39.875628+0000","last_fullsized":"2024-09-15T00:11:39.875628+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:19.680950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014193910000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6067,"num_read_kb":40333,"num_write":376,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"137'826","reported_seq":5711,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876271+0000","last_change":"2024-09-15T00:11:16.819173+0000","last_active":"2024-09-15T00:11:39.876271+0000","last_peered":"2024-09-15T00:11:39.876271+0000","last_clean":"2024-09-15T00:11:39.876271+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:11:39.876271+0000","last_undegraded":"2024-09-15T00:11:39.876271+0000","last_fullsized":"2024-09-15T00:11:39.876271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:38.089567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019789569999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8748,"num_read_kb":37428,"num_write":973,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"138'512","reported_seq":6131,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876263+0000","last_change":"2024-09-15T00:11:16.818860+0000","last_active":"2024-09-15T00:11:39.876263+0000","last_peered":"2024-09-15T00:11:39.876263+0000","last_clean":"2024-09-15T00:11:39.876263+0000","last_became_active":"2024-09-15T00:10:58.347433+0000","last_became_peered":"2024-09-15T00:10:58.347433+0000","last_unstale":"2024-09-15T00:11:39.876263+0000","last_undegraded":"2024-09-15T00:11:39.876263+0000","last_fullsized":"2024-09-15T00:11:39.876263+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:34.872358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878240000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6371,"num_read_kb":39648,"num_write":614,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"137'519","reported_seq":10491,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875469+0000","last_change":"2024-09-15T00:11:27.816441+0000","last_active":"2024-09-15T00:11:39.875469+0000","last_peered":"2024-09-15T00:11:39.875469+0000","last_clean":"2024-09-15T00:11:39.875469+0000","last_became_active":"2024-09-15T00:11:27.814209+0000","last_became_peered":"2024-09-15T00:11:27.814209+0000","last_unstale":"2024-09-15T00:11:39.875469+0000","last_undegraded":"2024-09-15T00:11:39.875469+0000","last_fullsized":"2024-09-15T00:11:39.875469+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:42.793496+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016246954000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11088,"num_read_kb":77879,"num_write":574,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"137'534","reported_seq":11124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797219+0000","last_change":"2024-09-15T00:11:16.807504+0000","last_active":"2024-09-15T00:11:40.797219+0000","last_peered":"2024-09-15T00:11:40.797219+0000","last_clean":"2024-09-15T00:11:40.797219+0000","last_became_active":"2024-09-15T00:10:58.346770+0000","last_became_peered":"2024-09-15T00:10:58.346770+0000","last_unstale":"2024-09-15T00:11:40.797219+0000","last_undegraded":"2024-09-15T00:11:40.797219+0000","last_fullsized":"2024-09-15T00:11:40.797219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:59.341684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015136839999999999,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":13910,"num_read_kb":40588,"num_write":444,"num_write_kb":8754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'709","reported_seq":10797,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457803+0000","last_change":"2024-09-15T00:11:16.811299+0000","last_active":"2024-09-15T00:11:40.457803+0000","last_peered":"2024-09-15T00:11:40.457803+0000","last_clean":"2024-09-15T00:11:40.457803+0000","last_became_active":"2024-09-15T00:10:58.350007+0000","last_became_peered":"2024-09-15T00:10:58.350007+0000","last_unstale":"2024-09-15T00:11:40.457803+0000","last_undegraded":"2024-09-15T00:11:40.457803+0000","last_fullsized":"2024-09-15T00:11:40.457803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:11.344098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019477310000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13597,"num_read_kb":55399,"num_write":535,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"137'655","reported_seq":8927,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875371+0000","last_change":"2024-09-15T00:11:28.815566+0000","last_active":"2024-09-15T00:11:39.875371+0000","last_peered":"2024-09-15T00:11:39.875371+0000","last_clean":"2024-09-15T00:11:39.875371+0000","last_became_active":"2024-09-15T00:11:28.813725+0000","last_became_peered":"2024-09-15T00:11:28.813725+0000","last_unstale":"2024-09-15T00:11:39.875371+0000","last_undegraded":"2024-09-15T00:11:39.875371+0000","last_fullsized":"2024-09-15T00:11:39.875371+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:28.982527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012280349999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9415,"num_read_kb":64591,"num_write":699,"num_write_kb":14239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"136'859","reported_seq":15047,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875360+0000","last_change":"2024-09-15T00:11:27.813449+0000","last_active":"2024-09-15T00:11:39.875360+0000","last_peered":"2024-09-15T00:11:39.875360+0000","last_clean":"2024-09-15T00:11:39.875360+0000","last_became_active":"2024-09-15T00:11:27.810983+0000","last_became_peered":"2024-09-15T00:11:27.810983+0000","last_unstale":"2024-09-15T00:11:39.875360+0000","last_undegraded":"2024-09-15T00:11:39.875360+0000","last_fullsized":"2024-09-15T00:11:39.875360+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:14.048930+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050753860000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":16900,"num_read_kb":72084,"num_write":902,"num_write_kb":44641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'832","reported_seq":8179,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876262+0000","last_change":"2024-09-15T00:11:16.818288+0000","last_active":"2024-09-15T00:11:39.876262+0000","last_peered":"2024-09-15T00:11:39.876262+0000","last_clean":"2024-09-15T00:11:39.876262+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:11:39.876262+0000","last_undegraded":"2024-09-15T00:11:39.876262+0000","last_fullsized":"2024-09-15T00:11:39.876262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.421381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098720300000000008,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10216,"num_read_kb":33960,"num_write":980,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"136'460","reported_seq":7066,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461827+0000","last_change":"2024-09-15T00:11:27.811672+0000","last_active":"2024-09-15T00:11:40.461827+0000","last_peered":"2024-09-15T00:11:40.461827+0000","last_clean":"2024-09-15T00:11:40.461827+0000","last_became_active":"2024-09-15T00:11:27.810534+0000","last_became_peered":"2024-09-15T00:11:27.810534+0000","last_unstale":"2024-09-15T00:11:40.461827+0000","last_undegraded":"2024-09-15T00:11:40.461827+0000","last_fullsized":"2024-09-15T00:11:40.461827+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:22:56.578888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081705799999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6309,"num_read_kb":62113,"num_write":514,"num_write_kb":29400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"136'577","reported_seq":8459,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461730+0000","last_change":"2024-09-15T00:11:16.822160+0000","last_active":"2024-09-15T00:11:40.461730+0000","last_peered":"2024-09-15T00:11:40.461730+0000","last_clean":"2024-09-15T00:11:40.461730+0000","last_became_active":"2024-09-15T00:10:58.353801+0000","last_became_peered":"2024-09-15T00:10:58.353801+0000","last_unstale":"2024-09-15T00:11:40.461730+0000","last_undegraded":"2024-09-15T00:11:40.461730+0000","last_fullsized":"2024-09-15T00:11:40.461730+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:56:52.432800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503938,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9435,"num_read_kb":45249,"num_write":397,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":310,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461963+0000","last_change":"2024-09-15T00:10:58.352604+0000","last_active":"2024-09-15T00:11:40.461963+0000","last_peered":"2024-09-15T00:11:40.461963+0000","last_clean":"2024-09-15T00:11:40.461963+0000","last_became_active":"2024-09-15T00:10:58.352314+0000","last_became_peered":"2024-09-15T00:10:58.352314+0000","last_unstale":"2024-09-15T00:11:40.461963+0000","last_undegraded":"2024-09-15T00:11:40.461963+0000","last_fullsized":"2024-09-15T00:11:40.461963+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:11.117426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"138'455","reported_seq":3568,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875353+0000","last_change":"2024-09-15T00:11:26.812244+0000","last_active":"2024-09-15T00:11:39.875353+0000","last_peered":"2024-09-15T00:11:39.875353+0000","last_clean":"2024-09-15T00:11:39.875353+0000","last_became_active":"2024-09-15T00:11:26.810222+0000","last_became_peered":"2024-09-15T00:11:26.810222+0000","last_unstale":"2024-09-15T00:11:39.875353+0000","last_undegraded":"2024-09-15T00:11:39.875353+0000","last_fullsized":"2024-09-15T00:11:39.875353+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:04:31.939139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099603400000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2834,"num_read_kb":21154,"num_write":404,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":302,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458045+0000","last_change":"2024-09-15T00:08:52.842938+0000","last_active":"2024-09-15T00:11:40.458045+0000","last_peered":"2024-09-15T00:11:40.458045+0000","last_clean":"2024-09-15T00:11:40.458045+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:11:40.458045+0000","last_undegraded":"2024-09-15T00:11:40.458045+0000","last_fullsized":"2024-09-15T00:11:40.458045+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:58.427419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'301","reported_seq":3604,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808663+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:52.093302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3137,"num_read_kb":36813,"num_write":322,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461876+0000","last_change":"2024-09-15T00:11:23.821084+0000","last_active":"2024-09-15T00:11:40.461876+0000","last_peered":"2024-09-15T00:11:40.461876+0000","last_clean":"2024-09-15T00:11:40.461876+0000","last_became_active":"2024-09-15T00:11:23.820814+0000","last_became_peered":"2024-09-15T00:11:23.820814+0000","last_unstale":"2024-09-15T00:11:40.461876+0000","last_undegraded":"2024-09-15T00:11:40.461876+0000","last_fullsized":"2024-09-15T00:11:40.461876+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:05:01.106813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"138'804","reported_seq":7267,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457816+0000","last_change":"2024-09-15T00:11:16.810988+0000","last_active":"2024-09-15T00:11:40.457816+0000","last_peered":"2024-09-15T00:11:40.457816+0000","last_clean":"2024-09-15T00:11:40.457816+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:11:40.457816+0000","last_undegraded":"2024-09-15T00:11:40.457816+0000","last_fullsized":"2024-09-15T00:11:40.457816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:19:00.332451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015707939999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9943,"num_read_kb":67757,"num_write":840,"num_write_kb":50219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":321,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457701+0000","last_change":"2024-09-15T00:11:23.823985+0000","last_active":"2024-09-15T00:11:40.457701+0000","last_peered":"2024-09-15T00:11:40.457701+0000","last_clean":"2024-09-15T00:11:40.457701+0000","last_became_active":"2024-09-15T00:11:23.823594+0000","last_became_peered":"2024-09-15T00:11:23.823594+0000","last_unstale":"2024-09-15T00:11:40.457701+0000","last_undegraded":"2024-09-15T00:11:40.457701+0000","last_fullsized":"2024-09-15T00:11:40.457701+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:01:04.517222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'598","reported_seq":5986,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876312+0000","last_change":"2024-09-15T00:11:16.819731+0000","last_active":"2024-09-15T00:11:39.876312+0000","last_peered":"2024-09-15T00:11:39.876312+0000","last_clean":"2024-09-15T00:11:39.876312+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:11:39.876312+0000","last_undegraded":"2024-09-15T00:11:39.876312+0000","last_fullsized":"2024-09-15T00:11:39.876312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:56.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022079199999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6563,"num_read_kb":34828,"num_write":657,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"138'669","reported_seq":6107,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797327+0000","last_change":"2024-09-15T00:11:26.813300+0000","last_active":"2024-09-15T00:11:40.797327+0000","last_peered":"2024-09-15T00:11:40.797327+0000","last_clean":"2024-09-15T00:11:40.797327+0000","last_became_active":"2024-09-15T00:11:26.812061+0000","last_became_peered":"2024-09-15T00:11:26.812061+0000","last_unstale":"2024-09-15T00:11:40.797327+0000","last_undegraded":"2024-09-15T00:11:40.797327+0000","last_fullsized":"2024-09-15T00:11:40.797327+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:59:00.972440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013113269999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7232,"num_read_kb":26564,"num_write":565,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876199+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:11:39.876199+0000","last_peered":"2024-09-15T00:11:39.876199+0000","last_clean":"2024-09-15T00:11:39.876199+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:11:39.876199+0000","last_undegraded":"2024-09-15T00:11:39.876199+0000","last_fullsized":"2024-09-15T00:11:39.876199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"136'714","reported_seq":7159,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457702+0000","last_change":"2024-09-15T00:11:16.811546+0000","last_active":"2024-09-15T00:11:40.457702+0000","last_peered":"2024-09-15T00:11:40.457702+0000","last_clean":"2024-09-15T00:11:40.457702+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:11:40.457702+0000","last_undegraded":"2024-09-15T00:11:40.457702+0000","last_fullsized":"2024-09-15T00:11:40.457702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:46.467190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570948,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7803,"num_read_kb":54109,"num_write":924,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875605+0000","last_change":"2024-09-15T00:08:52.836839+0000","last_active":"2024-09-15T00:11:39.875605+0000","last_peered":"2024-09-15T00:11:39.875605+0000","last_clean":"2024-09-15T00:11:39.875605+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:11:39.875605+0000","last_undegraded":"2024-09-15T00:11:39.875605+0000","last_fullsized":"2024-09-15T00:11:39.875605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:32.692248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'627","reported_seq":3714,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875090+0000","last_change":"2024-09-15T00:11:26.815022+0000","last_active":"2024-09-15T00:11:39.875090+0000","last_peered":"2024-09-15T00:11:39.875090+0000","last_clean":"2024-09-15T00:11:39.875090+0000","last_became_active":"2024-09-15T00:11:26.813953+0000","last_became_peered":"2024-09-15T00:11:26.813953+0000","last_unstale":"2024-09-15T00:11:39.875090+0000","last_undegraded":"2024-09-15T00:11:39.875090+0000","last_fullsized":"2024-09-15T00:11:39.875090+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:43:46.140096+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00071073200000000005,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5290,"num_read_kb":27132,"num_write":878,"num_write_kb":8037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2365199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":303,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461630+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:11:40.461630+0000","last_peered":"2024-09-15T00:11:40.461630+0000","last_clean":"2024-09-15T00:11:40.461630+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:11:40.461630+0000","last_undegraded":"2024-09-15T00:11:40.461630+0000","last_fullsized":"2024-09-15T00:11:40.461630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:36.933956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'493","reported_seq":6880,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458109+0000","last_change":"2024-09-15T00:11:16.810764+0000","last_active":"2024-09-15T00:11:40.458109+0000","last_peered":"2024-09-15T00:11:40.458109+0000","last_clean":"2024-09-15T00:11:40.458109+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:11:40.458109+0000","last_undegraded":"2024-09-15T00:11:40.458109+0000","last_fullsized":"2024-09-15T00:11:40.458109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:10:05.221091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215973,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6232,"num_read_kb":25476,"num_write":407,"num_write_kb":5284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":305,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876278+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:11:39.876278+0000","last_peered":"2024-09-15T00:11:39.876278+0000","last_clean":"2024-09-15T00:11:39.876278+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:11:39.876278+0000","last_undegraded":"2024-09-15T00:11:39.876278+0000","last_fullsized":"2024-09-15T00:11:39.876278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":482,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461680+0000","last_change":"2024-09-15T00:11:26.809502+0000","last_active":"2024-09-15T00:11:40.461680+0000","last_peered":"2024-09-15T00:11:40.461680+0000","last_clean":"2024-09-15T00:11:40.461680+0000","last_became_active":"2024-09-15T00:11:26.809081+0000","last_became_peered":"2024-09-15T00:11:26.809081+0000","last_unstale":"2024-09-15T00:11:40.461680+0000","last_undegraded":"2024-09-15T00:11:40.461680+0000","last_fullsized":"2024-09-15T00:11:40.461680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:33.161918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"136'570","reported_seq":11116,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875953+0000","last_change":"2024-09-15T00:11:16.808250+0000","last_active":"2024-09-15T00:11:39.875953+0000","last_peered":"2024-09-15T00:11:39.875953+0000","last_clean":"2024-09-15T00:11:39.875953+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:11:39.875953+0000","last_undegraded":"2024-09-15T00:11:39.875953+0000","last_fullsized":"2024-09-15T00:11:39.875953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:18.362309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098862600000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":12740,"num_read_kb":88117,"num_write":607,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"137'511","reported_seq":9033,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797237+0000","last_change":"2024-09-15T00:11:16.806418+0000","last_active":"2024-09-15T00:11:40.797237+0000","last_peered":"2024-09-15T00:11:40.797237+0000","last_clean":"2024-09-15T00:11:40.797237+0000","last_became_active":"2024-09-15T00:10:58.346994+0000","last_became_peered":"2024-09-15T00:10:58.346994+0000","last_unstale":"2024-09-15T00:11:40.797237+0000","last_undegraded":"2024-09-15T00:11:40.797237+0000","last_fullsized":"2024-09-15T00:11:40.797237+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:09:39.692848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088599499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9287,"num_read_kb":49680,"num_write":419,"num_write_kb":13143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"137'346","reported_seq":2534,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461768+0000","last_change":"2024-09-15T00:11:16.822630+0000","last_active":"2024-09-15T00:11:40.461768+0000","last_peered":"2024-09-15T00:11:40.461768+0000","last_clean":"2024-09-15T00:11:40.461768+0000","last_became_active":"2024-09-15T00:10:58.353160+0000","last_became_peered":"2024-09-15T00:10:58.353160+0000","last_unstale":"2024-09-15T00:11:40.461768+0000","last_undegraded":"2024-09-15T00:11:40.461768+0000","last_fullsized":"2024-09-15T00:11:40.461768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:32.587056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001573252,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2498,"num_read_kb":16284,"num_write":316,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"137'342","reported_seq":4084,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457821+0000","last_change":"2024-09-15T00:11:27.813271+0000","last_active":"2024-09-15T00:11:40.457821+0000","last_peered":"2024-09-15T00:11:40.457821+0000","last_clean":"2024-09-15T00:11:40.457821+0000","last_became_active":"2024-09-15T00:11:27.812092+0000","last_became_peered":"2024-09-15T00:11:27.812092+0000","last_unstale":"2024-09-15T00:11:40.457821+0000","last_undegraded":"2024-09-15T00:11:40.457821+0000","last_fullsized":"2024-09-15T00:11:40.457821+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:10:35.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000721333,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3470,"num_read_kb":33354,"num_write":301,"num_write_kb":6941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"138'1089","reported_seq":6155,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876331+0000","last_change":"2024-09-15T00:11:27.812382+0000","last_active":"2024-09-15T00:11:39.876331+0000","last_peered":"2024-09-15T00:11:39.876331+0000","last_clean":"2024-09-15T00:11:39.876331+0000","last_became_active":"2024-09-15T00:11:27.811047+0000","last_became_peered":"2024-09-15T00:11:27.811047+0000","last_unstale":"2024-09-15T00:11:39.876331+0000","last_undegraded":"2024-09-15T00:11:39.876331+0000","last_fullsized":"2024-09-15T00:11:39.876331+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1089,"log_dups_size":0,"ondisk_log_size":1089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:34.938354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021428929999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8751,"num_read_kb":31066,"num_write":1650,"num_write_kb":11591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'472","reported_seq":5184,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461560+0000","last_change":"2024-09-15T00:11:16.821809+0000","last_active":"2024-09-15T00:11:40.461560+0000","last_peered":"2024-09-15T00:11:40.461560+0000","last_clean":"2024-09-15T00:11:40.461560+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:11:40.461560+0000","last_undegraded":"2024-09-15T00:11:40.461560+0000","last_fullsized":"2024-09-15T00:11:40.461560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:55:03.400561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012186440000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6281,"num_read_kb":25421,"num_write":438,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"137'321","reported_seq":4493,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876475+0000","last_change":"2024-09-15T00:11:16.818687+0000","last_active":"2024-09-15T00:11:39.876475+0000","last_peered":"2024-09-15T00:11:39.876475+0000","last_clean":"2024-09-15T00:11:39.876475+0000","last_became_active":"2024-09-15T00:10:58.347480+0000","last_became_peered":"2024-09-15T00:10:58.347480+0000","last_unstale":"2024-09-15T00:11:39.876475+0000","last_undegraded":"2024-09-15T00:11:39.876475+0000","last_fullsized":"2024-09-15T00:11:39.876475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:57.538275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015201839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3806,"num_read_kb":22330,"num_write":293,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"135'777","reported_seq":5222,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876408+0000","last_change":"2024-09-15T00:11:16.819308+0000","last_active":"2024-09-15T00:11:39.876408+0000","last_peered":"2024-09-15T00:11:39.876408+0000","last_clean":"2024-09-15T00:11:39.876408+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:11:39.876408+0000","last_undegraded":"2024-09-15T00:11:39.876408+0000","last_fullsized":"2024-09-15T00:11:39.876408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:38.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018611719999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7063,"num_read_kb":22283,"num_write":819,"num_write_kb":6499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"136'345","reported_seq":6247,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876066+0000","last_change":"2024-09-15T00:11:26.810552+0000","last_active":"2024-09-15T00:11:39.876066+0000","last_peered":"2024-09-15T00:11:39.876066+0000","last_clean":"2024-09-15T00:11:39.876066+0000","last_became_active":"2024-09-15T00:11:26.809733+0000","last_became_peered":"2024-09-15T00:11:26.809733+0000","last_unstale":"2024-09-15T00:11:39.876066+0000","last_undegraded":"2024-09-15T00:11:39.876066+0000","last_fullsized":"2024-09-15T00:11:39.876066+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:03.405500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014647,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5082,"num_read_kb":32663,"num_write":383,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"134'1204","reported_seq":4832,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461699+0000","last_change":"2024-09-15T00:11:16.821784+0000","last_active":"2024-09-15T00:11:40.461699+0000","last_peered":"2024-09-15T00:11:40.461699+0000","last_clean":"2024-09-15T00:11:40.461699+0000","last_became_active":"2024-09-15T00:10:58.354658+0000","last_became_peered":"2024-09-15T00:10:58.354658+0000","last_unstale":"2024-09-15T00:11:40.461699+0000","last_undegraded":"2024-09-15T00:11:40.461699+0000","last_fullsized":"2024-09-15T00:11:40.461699+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:36.660704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185029,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7974,"num_read_kb":21277,"num_write":1815,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'1278","reported_seq":10322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875220+0000","last_change":"2024-09-15T00:11:27.815741+0000","last_active":"2024-09-15T00:11:39.875220+0000","last_peered":"2024-09-15T00:11:39.875220+0000","last_clean":"2024-09-15T00:11:39.875220+0000","last_became_active":"2024-09-15T00:11:27.813284+0000","last_became_peered":"2024-09-15T00:11:27.813284+0000","last_unstale":"2024-09-15T00:11:39.875220+0000","last_undegraded":"2024-09-15T00:11:39.875220+0000","last_fullsized":"2024-09-15T00:11:39.875220+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:58:44.187713+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010772937999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":12900,"num_read_kb":112467,"num_write":1928,"num_write_kb":43546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"137'243","reported_seq":2506,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461549+0000","last_change":"2024-09-15T00:11:16.822816+0000","last_active":"2024-09-15T00:11:40.461549+0000","last_peered":"2024-09-15T00:11:40.461549+0000","last_clean":"2024-09-15T00:11:40.461549+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:11:40.461549+0000","last_undegraded":"2024-09-15T00:11:40.461549+0000","last_fullsized":"2024-09-15T00:11:40.461549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:36:47.914784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013592509999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1867,"num_read_kb":15504,"num_write":229,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":79,"num_read":254890,"num_read_kb":1398144,"num_write":21738,"num_write_kb":428109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2462,"num_bytes_recovered":92274536,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26303,"internal_metadata":0},"log_size":19673,"ondisk_log_size":19673,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":27652,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59444,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312396,"statfs":{"total":96636764160,"available":96575893504,"internally_reserved":0,"allocated":1933312,"data_stored":811415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":118,"seq":506806140939,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34284,"kb_used_data":1876,"kb_used_omap":31,"kb_used_meta":32352,"kb_avail":94337556,"statfs":{"total":96636764160,"available":96601657344,"internally_reserved":0,"allocated":1921024,"data_stored":806387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31754,"internal_metadata":33129462},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81704,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290136,"statfs":{"total":96636764160,"available":96553099264,"internally_reserved":0,"allocated":1986560,"data_stored":1401764,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640296,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":1929216,"data_stored":811396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640296,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":811434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":133,"seq":571230650376,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39580,"kb_used_data":1924,"kb_used_omap":28,"kb_used_meta":37603,"kb_avail":94332260,"statfs":{"total":96636764160,"available":96596234240,"internally_reserved":0,"allocated":1970176,"data_stored":1401764,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28896,"internal_metadata":38506272},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15263,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11040,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:44.577 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T00:11:44.867 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:44.867 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":161,"fsid":"4f532d31-28c1-4236-a74e-6392960baf2b","created":"2024-09-15T00:08:32.007191+0000","modified":"2024-09-15T00:11:39.871757+0000","last_up_change":"2024-09-15T00:11:11.798639+0000","last_in_change":"2024-09-15T00:11:22.802027+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T00:08:42.896076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T00:08:49.173060+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-15T00:08:58.879058+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":43,"snap_epoch":138,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"56600dc2-b95d-47a7-b2b1-5ddc54c08341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":55,"up_from":133,"up_thru":149,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6817","nonce":1332878779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6819","nonce":1332878779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6823","nonce":1332878779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1332878779},{"type":"v1","addr":"172.21.15.27:6821","nonce":1332878779}]},"public_addr":"172.21.15.27:6817/1332878779","cluster_addr":"172.21.15.27:6819/1332878779","heartbeat_back_addr":"172.21.15.27:6823/1332878779","heartbeat_front_addr":"172.21.15.27:6821/1332878779","state":["exists","up"]},{"osd":1,"uuid":"ea8f0155-8472-4e87-89fd-7716d05065d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6809","nonce":3601510193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6811","nonce":3601510193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6815","nonce":3601510193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3601510193},{"type":"v1","addr":"172.21.15.27:6813","nonce":3601510193}]},"public_addr":"172.21.15.27:6809/3601510193","cluster_addr":"172.21.15.27:6811/3601510193","heartbeat_back_addr":"172.21.15.27:6815/3601510193","heartbeat_front_addr":"172.21.15.27:6813/3601510193","state":["exists","up"]},{"osd":2,"uuid":"357d2d59-66f0-47b8-87c8-4acdfe62a07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6801","nonce":3222759256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6803","nonce":3222759256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6807","nonce":3222759256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3222759256},{"type":"v1","addr":"172.21.15.27:6805","nonce":3222759256}]},"public_addr":"172.21.15.27:6801/3222759256","cluster_addr":"172.21.15.27:6803/3222759256","heartbeat_back_addr":"172.21.15.27:6807/3222759256","heartbeat_front_addr":"172.21.15.27:6805/3222759256","state":["exists","up"]},{"osd":3,"uuid":"22208f56-d8aa-4a56-a42c-5a990962b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6809","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6811","nonce":675428777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6813","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6816","nonce":675428777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6823","nonce":675428777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":675428777},{"type":"v1","addr":"172.21.15.79:6821","nonce":675428777}]},"public_addr":"172.21.15.79:6811/675428777","cluster_addr":"172.21.15.79:6816/675428777","heartbeat_back_addr":"172.21.15.79:6823/675428777","heartbeat_front_addr":"172.21.15.79:6821/675428777","state":["exists","up"]},{"osd":4,"uuid":"a974629e-9ef0-40a3-94bd-382e0bfa8bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":88,"up_from":118,"up_thru":147,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6810","nonce":1046239476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6814","nonce":1046239476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6819","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6820","nonce":1046239476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6815","nonce":1046239476},{"type":"v1","addr":"172.21.15.79:6817","nonce":1046239476}]},"public_addr":"172.21.15.79:6810/1046239476","cluster_addr":"172.21.15.79:6814/1046239476","heartbeat_back_addr":"172.21.15.79:6820/1046239476","heartbeat_front_addr":"172.21.15.79:6817/1046239476","state":["exists","up"]},{"osd":5,"uuid":"06b73c2d-466a-4c4e-89fa-31d6eeb1af1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6801","nonce":2235671634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6803","nonce":2235671634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6807","nonce":2235671634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2235671634},{"type":"v1","addr":"172.21.15.79:6805","nonce":2235671634}]},"public_addr":"172.21.15.79:6801/2235671634","cluster_addr":"172.21.15.79:6803/2235671634","heartbeat_back_addr":"172.21.15.79:6807/2235671634","heartbeat_front_addr":"172.21.15.79:6805/2235671634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.730107+0000","dead_epoch":56},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.365297+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.868500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:40.788781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.892143+0000","dead_epoch":90},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T00:08:39.990037+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.12","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T00:11:45.881 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T00:11:45.881 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:11:46.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T00:11:46.201+0000 7efcc89ba640 0 --2- 172.21.15.27:0/578397556 >> [v2:172.21.15.27:3301/0,v1:172.21.15.27:6790/0] conn(0x7efcc4099a20 0x7efcc4099e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T00:11:46.223 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:46.223 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:11:46.223 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:46.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T00:11:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T00:11:46.538 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T00:11:46.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:11:46.664 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:11:46.664 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:46.674 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T00:11:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T00:11:46.980 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T00:11:46.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:11:47.105 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:47.105 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:11:47.105 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:47.116 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T00:11:47.402 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T00:11:47.414 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T00:11:47.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:11:47.542 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:47.542 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:11:47.542 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:47.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T00:11:47.842 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T00:11:47.855 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T00:11:47.855 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:11:47.982 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:11:47.982 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:47.993 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T00:11:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T00:11:48.294 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T00:11:48.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T00:11:48.419 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T00:11:48.419 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T00:11:48.420 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T00:11:48.430 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T00:11:48.726 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T00:11:48.738 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:11:49.032 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:11:49.033 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:11:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-15T00:11:48.884870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":79,"num_read":255006,"num_read_kb":1398244,"num_write":21899,"num_write_kb":430527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2466,"num_bytes_recovered":93455272,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19797,"ondisk_log_size":19797,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":311696,"kb_used_data":11416,"kb_used_omap":65,"kb_used_meta":300094,"kb_avail":565919344,"statfs":{"total":579820584960,"available":579501408256,"internally_reserved":0,"allocated":11689984,"data_stored":6049188,"data_compressed":18090,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":67028,"internal_metadata":307296812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002251"},"pg_stats":[{"pgid":"3.1f","version":"138'535","reported_seq":11588,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875301+0000","last_change":"2024-09-15T00:11:28.816240+0000","last_active":"2024-09-15T00:11:39.875301+0000","last_peered":"2024-09-15T00:11:39.875301+0000","last_clean":"2024-09-15T00:11:39.875301+0000","last_became_active":"2024-09-15T00:11:28.814325+0000","last_became_peered":"2024-09-15T00:11:28.814325+0000","last_unstale":"2024-09-15T00:11:39.875301+0000","last_undegraded":"2024-09-15T00:11:39.875301+0000","last_fullsized":"2024-09-15T00:11:39.875301+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001206129,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12081,"num_read_kb":64593,"num_write":535,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"135'496","reported_seq":5532,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875677+0000","last_change":"2024-09-15T00:11:16.808902+0000","last_active":"2024-09-15T00:11:39.875677+0000","last_peered":"2024-09-15T00:11:39.875677+0000","last_clean":"2024-09-15T00:11:39.875677+0000","last_became_active":"2024-09-15T00:10:58.347628+0000","last_became_peered":"2024-09-15T00:10:58.347628+0000","last_unstale":"2024-09-15T00:11:39.875677+0000","last_undegraded":"2024-09-15T00:11:39.875677+0000","last_fullsized":"2024-09-15T00:11:39.875677+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014193910000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6067,"num_read_kb":40333,"num_write":376,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"137'826","reported_seq":5711,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876271+0000","last_change":"2024-09-15T00:11:16.819173+0000","last_active":"2024-09-15T00:11:39.876271+0000","last_peered":"2024-09-15T00:11:39.876271+0000","last_clean":"2024-09-15T00:11:39.876271+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:11:39.876271+0000","last_undegraded":"2024-09-15T00:11:39.876271+0000","last_fullsized":"2024-09-15T00:11:39.876271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:38.089567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019789569999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8748,"num_read_kb":37428,"num_write":973,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"138'512","reported_seq":6131,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876263+0000","last_change":"2024-09-15T00:11:16.818860+0000","last_active":"2024-09-15T00:11:39.876263+0000","last_peered":"2024-09-15T00:11:39.876263+0000","last_clean":"2024-09-15T00:11:39.876263+0000","last_became_active":"2024-09-15T00:10:58.347433+0000","last_became_peered":"2024-09-15T00:10:58.347433+0000","last_unstale":"2024-09-15T00:11:39.876263+0000","last_undegraded":"2024-09-15T00:11:39.876263+0000","last_fullsized":"2024-09-15T00:11:39.876263+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:34.872358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878240000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6371,"num_read_kb":39648,"num_write":614,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"137'519","reported_seq":10492,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875511+0000","last_change":"2024-09-15T00:11:27.816441+0000","last_active":"2024-09-15T00:11:39.875511+0000","last_peered":"2024-09-15T00:11:39.875511+0000","last_clean":"2024-09-15T00:11:39.875511+0000","last_became_active":"2024-09-15T00:11:27.814209+0000","last_became_peered":"2024-09-15T00:11:27.814209+0000","last_unstale":"2024-09-15T00:11:39.875511+0000","last_undegraded":"2024-09-15T00:11:39.875511+0000","last_fullsized":"2024-09-15T00:11:39.875511+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016246954000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11088,"num_read_kb":77879,"num_write":574,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"137'534","reported_seq":11124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797219+0000","last_change":"2024-09-15T00:11:16.807504+0000","last_active":"2024-09-15T00:11:40.797219+0000","last_peered":"2024-09-15T00:11:40.797219+0000","last_clean":"2024-09-15T00:11:40.797219+0000","last_became_active":"2024-09-15T00:10:58.346770+0000","last_became_peered":"2024-09-15T00:10:58.346770+0000","last_unstale":"2024-09-15T00:11:40.797219+0000","last_undegraded":"2024-09-15T00:11:40.797219+0000","last_fullsized":"2024-09-15T00:11:40.797219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:16:59.341684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015136839999999999,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":13910,"num_read_kb":40588,"num_write":444,"num_write_kb":8754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'709","reported_seq":10798,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457845+0000","last_change":"2024-09-15T00:11:16.811299+0000","last_active":"2024-09-15T00:11:40.457845+0000","last_peered":"2024-09-15T00:11:40.457845+0000","last_clean":"2024-09-15T00:11:40.457845+0000","last_became_active":"2024-09-15T00:10:58.350007+0000","last_became_peered":"2024-09-15T00:10:58.350007+0000","last_unstale":"2024-09-15T00:11:40.457845+0000","last_undegraded":"2024-09-15T00:11:40.457845+0000","last_fullsized":"2024-09-15T00:11:40.457845+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019477310000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13597,"num_read_kb":55399,"num_write":535,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"137'655","reported_seq":8928,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875430+0000","last_change":"2024-09-15T00:11:28.815566+0000","last_active":"2024-09-15T00:11:39.875430+0000","last_peered":"2024-09-15T00:11:39.875430+0000","last_clean":"2024-09-15T00:11:39.875430+0000","last_became_active":"2024-09-15T00:11:28.813725+0000","last_became_peered":"2024-09-15T00:11:28.813725+0000","last_unstale":"2024-09-15T00:11:39.875430+0000","last_undegraded":"2024-09-15T00:11:39.875430+0000","last_fullsized":"2024-09-15T00:11:39.875430+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012280349999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9415,"num_read_kb":64591,"num_write":699,"num_write_kb":14239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"136'859","reported_seq":15048,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875409+0000","last_change":"2024-09-15T00:11:27.813449+0000","last_active":"2024-09-15T00:11:39.875409+0000","last_peered":"2024-09-15T00:11:39.875409+0000","last_clean":"2024-09-15T00:11:39.875409+0000","last_became_active":"2024-09-15T00:11:27.810983+0000","last_became_peered":"2024-09-15T00:11:27.810983+0000","last_unstale":"2024-09-15T00:11:39.875409+0000","last_undegraded":"2024-09-15T00:11:39.875409+0000","last_fullsized":"2024-09-15T00:11:39.875409+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0050753860000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":16900,"num_read_kb":72084,"num_write":902,"num_write_kb":44641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'832","reported_seq":8179,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876262+0000","last_change":"2024-09-15T00:11:16.818288+0000","last_active":"2024-09-15T00:11:39.876262+0000","last_peered":"2024-09-15T00:11:39.876262+0000","last_clean":"2024-09-15T00:11:39.876262+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:11:39.876262+0000","last_undegraded":"2024-09-15T00:11:39.876262+0000","last_fullsized":"2024-09-15T00:11:39.876262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.421381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098720300000000008,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10216,"num_read_kb":33960,"num_write":980,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"136'460","reported_seq":7067,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461881+0000","last_change":"2024-09-15T00:11:27.811672+0000","last_active":"2024-09-15T00:11:40.461881+0000","last_peered":"2024-09-15T00:11:40.461881+0000","last_clean":"2024-09-15T00:11:40.461881+0000","last_became_active":"2024-09-15T00:11:27.810534+0000","last_became_peered":"2024-09-15T00:11:27.810534+0000","last_unstale":"2024-09-15T00:11:40.461881+0000","last_undegraded":"2024-09-15T00:11:40.461881+0000","last_fullsized":"2024-09-15T00:11:40.461881+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081705799999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6309,"num_read_kb":62113,"num_write":514,"num_write_kb":29400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"136'577","reported_seq":8460,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461787+0000","last_change":"2024-09-15T00:11:16.822160+0000","last_active":"2024-09-15T00:11:40.461787+0000","last_peered":"2024-09-15T00:11:40.461787+0000","last_clean":"2024-09-15T00:11:40.461787+0000","last_became_active":"2024-09-15T00:10:58.353801+0000","last_became_peered":"2024-09-15T00:10:58.353801+0000","last_unstale":"2024-09-15T00:11:40.461787+0000","last_undegraded":"2024-09-15T00:11:40.461787+0000","last_fullsized":"2024-09-15T00:11:40.461787+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001503938,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9435,"num_read_kb":45249,"num_write":397,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":311,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461994+0000","last_change":"2024-09-15T00:10:58.352604+0000","last_active":"2024-09-15T00:11:40.461994+0000","last_peered":"2024-09-15T00:11:40.461994+0000","last_clean":"2024-09-15T00:11:40.461994+0000","last_became_active":"2024-09-15T00:10:58.352314+0000","last_became_peered":"2024-09-15T00:10:58.352314+0000","last_unstale":"2024-09-15T00:11:40.461994+0000","last_undegraded":"2024-09-15T00:11:40.461994+0000","last_fullsized":"2024-09-15T00:11:40.461994+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"161'459","reported_seq":3581,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:46.798463+0000","last_change":"2024-09-15T00:11:46.798515+0000","last_active":"2024-09-15T00:11:46.798463+0000","last_peered":"2024-09-15T00:11:46.798463+0000","last_clean":"2024-09-15T00:11:46.798463+0000","last_became_active":"2024-09-15T00:11:26.810222+0000","last_became_peered":"2024-09-15T00:11:26.810222+0000","last_unstale":"2024-09-15T00:11:46.798463+0000","last_undegraded":"2024-09-15T00:11:46.798463+0000","last_fullsized":"2024-09-15T00:11:46.798463+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'459","last_scrub_stamp":"2024-09-15T00:11:46.798335+0000","last_deep_scrub":"161'459","last_deep_scrub_stamp":"2024-09-15T00:11:46.798335+0000","last_clean_scrub_stamp":"2024-09-15T00:11:46.798335+0000","objects_scrubbed":4,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:53:00.877802+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00099603400000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2834,"num_read_kb":21154,"num_write":404,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":311,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:47.481287+0000","last_change":"2024-09-15T00:11:47.481307+0000","last_active":"2024-09-15T00:11:47.481287+0000","last_peered":"2024-09-15T00:11:47.481287+0000","last_clean":"2024-09-15T00:11:47.481287+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:11:47.481287+0000","last_undegraded":"2024-09-15T00:11:47.481287+0000","last_fullsized":"2024-09-15T00:11:47.481287+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:47.481210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:47.481210+0000","last_clean_scrub_stamp":"2024-09-15T00:11:47.481210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:10.788853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'301","reported_seq":3605,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876025+0000","last_change":"2024-09-15T00:11:16.808663+0000","last_active":"2024-09-15T00:11:39.876025+0000","last_peered":"2024-09-15T00:11:39.876025+0000","last_clean":"2024-09-15T00:11:39.876025+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:11:39.876025+0000","last_undegraded":"2024-09-15T00:11:39.876025+0000","last_fullsized":"2024-09-15T00:11:39.876025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3137,"num_read_kb":36813,"num_write":322,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":323,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461955+0000","last_change":"2024-09-15T00:11:23.821084+0000","last_active":"2024-09-15T00:11:40.461955+0000","last_peered":"2024-09-15T00:11:40.461955+0000","last_clean":"2024-09-15T00:11:40.461955+0000","last_became_active":"2024-09-15T00:11:23.820814+0000","last_became_peered":"2024-09-15T00:11:23.820814+0000","last_unstale":"2024-09-15T00:11:40.461955+0000","last_undegraded":"2024-09-15T00:11:40.461955+0000","last_fullsized":"2024-09-15T00:11:40.461955+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"138'804","reported_seq":7268,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457859+0000","last_change":"2024-09-15T00:11:16.810988+0000","last_active":"2024-09-15T00:11:40.457859+0000","last_peered":"2024-09-15T00:11:40.457859+0000","last_clean":"2024-09-15T00:11:40.457859+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:11:40.457859+0000","last_undegraded":"2024-09-15T00:11:40.457859+0000","last_fullsized":"2024-09-15T00:11:40.457859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015707939999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9943,"num_read_kb":67757,"num_write":840,"num_write_kb":50219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":322,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457750+0000","last_change":"2024-09-15T00:11:23.823985+0000","last_active":"2024-09-15T00:11:40.457750+0000","last_peered":"2024-09-15T00:11:40.457750+0000","last_clean":"2024-09-15T00:11:40.457750+0000","last_became_active":"2024-09-15T00:11:23.823594+0000","last_became_peered":"2024-09-15T00:11:23.823594+0000","last_unstale":"2024-09-15T00:11:40.457750+0000","last_undegraded":"2024-09-15T00:11:40.457750+0000","last_fullsized":"2024-09-15T00:11:40.457750+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'598","reported_seq":5986,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876312+0000","last_change":"2024-09-15T00:11:16.819731+0000","last_active":"2024-09-15T00:11:39.876312+0000","last_peered":"2024-09-15T00:11:39.876312+0000","last_clean":"2024-09-15T00:11:39.876312+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:11:39.876312+0000","last_undegraded":"2024-09-15T00:11:39.876312+0000","last_fullsized":"2024-09-15T00:11:39.876312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:28:56.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022079199999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6563,"num_read_kb":34828,"num_write":657,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"138'669","reported_seq":6107,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797327+0000","last_change":"2024-09-15T00:11:26.813300+0000","last_active":"2024-09-15T00:11:40.797327+0000","last_peered":"2024-09-15T00:11:40.797327+0000","last_clean":"2024-09-15T00:11:40.797327+0000","last_became_active":"2024-09-15T00:11:26.812061+0000","last_became_peered":"2024-09-15T00:11:26.812061+0000","last_unstale":"2024-09-15T00:11:40.797327+0000","last_undegraded":"2024-09-15T00:11:40.797327+0000","last_fullsized":"2024-09-15T00:11:40.797327+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:59:00.972440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013113269999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7232,"num_read_kb":26564,"num_write":565,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":306,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876199+0000","last_change":"2024-09-15T00:08:52.840013+0000","last_active":"2024-09-15T00:11:39.876199+0000","last_peered":"2024-09-15T00:11:39.876199+0000","last_clean":"2024-09-15T00:11:39.876199+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:11:39.876199+0000","last_undegraded":"2024-09-15T00:11:39.876199+0000","last_fullsized":"2024-09-15T00:11:39.876199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:43.496440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"136'714","reported_seq":7160,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457938+0000","last_change":"2024-09-15T00:11:16.811546+0000","last_active":"2024-09-15T00:11:40.457938+0000","last_peered":"2024-09-15T00:11:40.457938+0000","last_clean":"2024-09-15T00:11:40.457938+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:11:40.457938+0000","last_undegraded":"2024-09-15T00:11:40.457938+0000","last_fullsized":"2024-09-15T00:11:40.457938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001570948,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7803,"num_read_kb":54109,"num_write":924,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":315,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:47.532626+0000","last_change":"2024-09-15T00:11:47.532650+0000","last_active":"2024-09-15T00:11:47.532626+0000","last_peered":"2024-09-15T00:11:47.532626+0000","last_clean":"2024-09-15T00:11:47.532626+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:11:47.532626+0000","last_undegraded":"2024-09-15T00:11:47.532626+0000","last_fullsized":"2024-09-15T00:11:47.532626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:47.532536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:47.532536+0000","last_clean_scrub_stamp":"2024-09-15T00:11:47.532536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:42:56.028867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'627","reported_seq":3715,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875146+0000","last_change":"2024-09-15T00:11:26.815022+0000","last_active":"2024-09-15T00:11:39.875146+0000","last_peered":"2024-09-15T00:11:39.875146+0000","last_clean":"2024-09-15T00:11:39.875146+0000","last_became_active":"2024-09-15T00:11:26.813953+0000","last_became_peered":"2024-09-15T00:11:26.813953+0000","last_unstale":"2024-09-15T00:11:39.875146+0000","last_undegraded":"2024-09-15T00:11:39.875146+0000","last_fullsized":"2024-09-15T00:11:39.875146+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00071073200000000005,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5290,"num_read_kb":27132,"num_write":878,"num_write_kb":8037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2365199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":304,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461702+0000","last_change":"2024-09-15T00:08:52.835672+0000","last_active":"2024-09-15T00:11:40.461702+0000","last_peered":"2024-09-15T00:11:40.461702+0000","last_clean":"2024-09-15T00:11:40.461702+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:11:40.461702+0000","last_undegraded":"2024-09-15T00:11:40.461702+0000","last_fullsized":"2024-09-15T00:11:40.461702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'493","reported_seq":6881,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.458129+0000","last_change":"2024-09-15T00:11:16.810764+0000","last_active":"2024-09-15T00:11:40.458129+0000","last_peered":"2024-09-15T00:11:40.458129+0000","last_clean":"2024-09-15T00:11:40.458129+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:11:40.458129+0000","last_undegraded":"2024-09-15T00:11:40.458129+0000","last_fullsized":"2024-09-15T00:11:40.458129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215973,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6232,"num_read_kb":25476,"num_write":407,"num_write_kb":5284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":305,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876278+0000","last_change":"2024-09-15T00:08:52.839513+0000","last_active":"2024-09-15T00:11:39.876278+0000","last_peered":"2024-09-15T00:11:39.876278+0000","last_clean":"2024-09-15T00:11:39.876278+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:11:39.876278+0000","last_undegraded":"2024-09-15T00:11:39.876278+0000","last_fullsized":"2024-09-15T00:11:39.876278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:49.813081+0000","last_clean_scrub_stamp":"2024-09-15T00:08:49.813081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:30.079741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":483,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461733+0000","last_change":"2024-09-15T00:11:26.809502+0000","last_active":"2024-09-15T00:11:40.461733+0000","last_peered":"2024-09-15T00:11:40.461733+0000","last_clean":"2024-09-15T00:11:40.461733+0000","last_became_active":"2024-09-15T00:11:26.809081+0000","last_became_peered":"2024-09-15T00:11:26.809081+0000","last_unstale":"2024-09-15T00:11:40.461733+0000","last_undegraded":"2024-09-15T00:11:40.461733+0000","last_fullsized":"2024-09-15T00:11:40.461733+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:43.796175+0000","last_clean_scrub_stamp":"2024-09-15T00:08:43.796175+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"136'570","reported_seq":11117,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876003+0000","last_change":"2024-09-15T00:11:16.808250+0000","last_active":"2024-09-15T00:11:39.876003+0000","last_peered":"2024-09-15T00:11:39.876003+0000","last_clean":"2024-09-15T00:11:39.876003+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:11:39.876003+0000","last_undegraded":"2024-09-15T00:11:39.876003+0000","last_fullsized":"2024-09-15T00:11:39.876003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098862600000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":12740,"num_read_kb":88117,"num_write":607,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"137'511","reported_seq":9033,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.797237+0000","last_change":"2024-09-15T00:11:16.806418+0000","last_active":"2024-09-15T00:11:40.797237+0000","last_peered":"2024-09-15T00:11:40.797237+0000","last_clean":"2024-09-15T00:11:40.797237+0000","last_became_active":"2024-09-15T00:10:58.346994+0000","last_became_peered":"2024-09-15T00:10:58.346994+0000","last_unstale":"2024-09-15T00:11:40.797237+0000","last_undegraded":"2024-09-15T00:11:40.797237+0000","last_fullsized":"2024-09-15T00:11:40.797237+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:09:39.692848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088599499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9287,"num_read_kb":49680,"num_write":419,"num_write_kb":13143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"137'346","reported_seq":2535,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461829+0000","last_change":"2024-09-15T00:11:16.822630+0000","last_active":"2024-09-15T00:11:40.461829+0000","last_peered":"2024-09-15T00:11:40.461829+0000","last_clean":"2024-09-15T00:11:40.461829+0000","last_became_active":"2024-09-15T00:10:58.353160+0000","last_became_peered":"2024-09-15T00:10:58.353160+0000","last_unstale":"2024-09-15T00:11:40.461829+0000","last_undegraded":"2024-09-15T00:11:40.461829+0000","last_fullsized":"2024-09-15T00:11:40.461829+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001573252,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2498,"num_read_kb":16284,"num_write":316,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"137'342","reported_seq":4085,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.457862+0000","last_change":"2024-09-15T00:11:27.813271+0000","last_active":"2024-09-15T00:11:40.457862+0000","last_peered":"2024-09-15T00:11:40.457862+0000","last_clean":"2024-09-15T00:11:40.457862+0000","last_became_active":"2024-09-15T00:11:27.812092+0000","last_became_peered":"2024-09-15T00:11:27.812092+0000","last_unstale":"2024-09-15T00:11:40.457862+0000","last_undegraded":"2024-09-15T00:11:40.457862+0000","last_fullsized":"2024-09-15T00:11:40.457862+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000721333,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3470,"num_read_kb":33354,"num_write":301,"num_write_kb":6941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"138'1089","reported_seq":6155,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876331+0000","last_change":"2024-09-15T00:11:27.812382+0000","last_active":"2024-09-15T00:11:39.876331+0000","last_peered":"2024-09-15T00:11:39.876331+0000","last_clean":"2024-09-15T00:11:39.876331+0000","last_became_active":"2024-09-15T00:11:27.811047+0000","last_became_peered":"2024-09-15T00:11:27.811047+0000","last_unstale":"2024-09-15T00:11:39.876331+0000","last_undegraded":"2024-09-15T00:11:39.876331+0000","last_fullsized":"2024-09-15T00:11:39.876331+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1089,"log_dups_size":0,"ondisk_log_size":1089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:34.938354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021428929999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8751,"num_read_kb":31066,"num_write":1650,"num_write_kb":11591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"137'472","reported_seq":5185,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461612+0000","last_change":"2024-09-15T00:11:16.821809+0000","last_active":"2024-09-15T00:11:40.461612+0000","last_peered":"2024-09-15T00:11:40.461612+0000","last_clean":"2024-09-15T00:11:40.461612+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:11:40.461612+0000","last_undegraded":"2024-09-15T00:11:40.461612+0000","last_fullsized":"2024-09-15T00:11:40.461612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012186440000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6281,"num_read_kb":25421,"num_write":438,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"137'321","reported_seq":4493,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876475+0000","last_change":"2024-09-15T00:11:16.818687+0000","last_active":"2024-09-15T00:11:39.876475+0000","last_peered":"2024-09-15T00:11:39.876475+0000","last_clean":"2024-09-15T00:11:39.876475+0000","last_became_active":"2024-09-15T00:10:58.347480+0000","last_became_peered":"2024-09-15T00:10:58.347480+0000","last_unstale":"2024-09-15T00:11:39.876475+0000","last_undegraded":"2024-09-15T00:11:39.876475+0000","last_fullsized":"2024-09-15T00:11:39.876475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:57.538275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015201839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3806,"num_read_kb":22330,"num_write":293,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"135'777","reported_seq":5222,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876408+0000","last_change":"2024-09-15T00:11:16.819308+0000","last_active":"2024-09-15T00:11:39.876408+0000","last_peered":"2024-09-15T00:11:39.876408+0000","last_clean":"2024-09-15T00:11:39.876408+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:11:39.876408+0000","last_undegraded":"2024-09-15T00:11:39.876408+0000","last_fullsized":"2024-09-15T00:11:39.876408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:38.990588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018611719999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7063,"num_read_kb":22283,"num_write":819,"num_write_kb":6499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"136'345","reported_seq":6247,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.876066+0000","last_change":"2024-09-15T00:11:26.810552+0000","last_active":"2024-09-15T00:11:39.876066+0000","last_peered":"2024-09-15T00:11:39.876066+0000","last_clean":"2024-09-15T00:11:39.876066+0000","last_became_active":"2024-09-15T00:11:26.809733+0000","last_became_peered":"2024-09-15T00:11:26.809733+0000","last_unstale":"2024-09-15T00:11:39.876066+0000","last_undegraded":"2024-09-15T00:11:39.876066+0000","last_fullsized":"2024-09-15T00:11:39.876066+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:03.405500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014647,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5082,"num_read_kb":32663,"num_write":383,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"134'1204","reported_seq":4833,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461752+0000","last_change":"2024-09-15T00:11:16.821784+0000","last_active":"2024-09-15T00:11:40.461752+0000","last_peered":"2024-09-15T00:11:40.461752+0000","last_clean":"2024-09-15T00:11:40.461752+0000","last_became_active":"2024-09-15T00:10:58.354658+0000","last_became_peered":"2024-09-15T00:10:58.354658+0000","last_unstale":"2024-09-15T00:11:40.461752+0000","last_undegraded":"2024-09-15T00:11:40.461752+0000","last_fullsized":"2024-09-15T00:11:40.461752+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185029,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7974,"num_read_kb":21277,"num_write":1815,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'1278","reported_seq":10323,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:39.875266+0000","last_change":"2024-09-15T00:11:27.815741+0000","last_active":"2024-09-15T00:11:39.875266+0000","last_peered":"2024-09-15T00:11:39.875266+0000","last_clean":"2024-09-15T00:11:39.875266+0000","last_became_active":"2024-09-15T00:11:27.813284+0000","last_became_peered":"2024-09-15T00:11:27.813284+0000","last_unstale":"2024-09-15T00:11:39.875266+0000","last_undegraded":"2024-09-15T00:11:39.875266+0000","last_fullsized":"2024-09-15T00:11:39.875266+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010772937999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":12900,"num_read_kb":112467,"num_write":1928,"num_write_kb":43546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"137'243","reported_seq":2507,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:40.461612+0000","last_change":"2024-09-15T00:11:16.822816+0000","last_active":"2024-09-15T00:11:40.461612+0000","last_peered":"2024-09-15T00:11:40.461612+0000","last_clean":"2024-09-15T00:11:40.461612+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:11:40.461612+0000","last_undegraded":"2024-09-15T00:11:40.461612+0000","last_fullsized":"2024-09-15T00:11:40.461612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:08:58.929182+0000","last_clean_scrub_stamp":"2024-09-15T00:08:58.929182+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013592509999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1867,"num_read_kb":15504,"num_write":229,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":79,"num_read":254890,"num_read_kb":1398144,"num_write":21738,"num_write_kb":428109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2462,"num_bytes_recovered":92274536,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26303,"internal_metadata":0},"log_size":19677,"ondisk_log_size":19677,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":27652,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59444,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312396,"statfs":{"total":96636764160,"available":96575893504,"internally_reserved":0,"allocated":1933312,"data_stored":811415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":118,"seq":506806140940,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34292,"kb_used_data":1888,"kb_used_omap":31,"kb_used_meta":32352,"kb_avail":94337548,"statfs":{"total":96636764160,"available":96601649152,"internally_reserved":0,"allocated":1933312,"data_stored":811415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31754,"internal_metadata":33129462},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81704,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290136,"statfs":{"total":96636764160,"available":96553099264,"internally_reserved":0,"allocated":1986560,"data_stored":1401764,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640297,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":1929216,"data_stored":811396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640297,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1937408,"data_stored":811434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":133,"seq":571230650377,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39580,"kb_used_data":1924,"kb_used_omap":28,"kb_used_meta":37603,"kb_avail":94332260,"statfs":{"total":96636764160,"available":96596234240,"internally_reserved":0,"allocated":1970176,"data_stored":1401764,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28896,"internal_metadata":38506272},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15263,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11040,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:11:49.049 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.049 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.049 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.049 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.049 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.049 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.049 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.049 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.050 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.050 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.050 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.050 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.050 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T00:08:49.813081+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.051 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.051 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T00:08:49.813081+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.051 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.051 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T00:08:49.813081+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.051 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.051 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.051 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T00:08:49.813081+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.051 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.051 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.052 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T00:08:49.813081+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.052 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.052 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T00:08:49.813081+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.052 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T00:08:43.796175+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.052 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.052 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.052 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.052 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.053 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.053 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.053 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.053 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.053 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.053 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.054 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.054 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-15T00:08:58.929182+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=8, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=0, tm_min=11, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T00:11:49.054 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T00:12:09.055 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:12:09.358 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T00:12:09.358 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T00:12:09.370 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-15T00:12:08.889232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":79,"num_read":255006,"num_read_kb":1398244,"num_write":21899,"num_write_kb":430527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2466,"num_bytes_recovered":93455272,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19880,"ondisk_log_size":19880,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":311672,"kb_used_data":11416,"kb_used_omap":65,"kb_used_meta":300094,"kb_avail":565919368,"statfs":{"total":579820584960,"available":579501432832,"internally_reserved":0,"allocated":11689984,"data_stored":6049188,"data_compressed":18090,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":67028,"internal_metadata":307296812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002584"},"pg_stats":[{"pgid":"3.1f","version":"161'536","reported_seq":11597,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:51.795587+0000","last_change":"2024-09-15T00:11:51.795619+0000","last_active":"2024-09-15T00:11:51.795587+0000","last_peered":"2024-09-15T00:11:51.795587+0000","last_clean":"2024-09-15T00:11:51.795587+0000","last_became_active":"2024-09-15T00:11:28.814325+0000","last_became_peered":"2024-09-15T00:11:28.814325+0000","last_unstale":"2024-09-15T00:11:51.795587+0000","last_undegraded":"2024-09-15T00:11:51.795587+0000","last_fullsized":"2024-09-15T00:11:51.795587+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'536","last_scrub_stamp":"2024-09-15T00:11:51.795493+0000","last_deep_scrub":"161'536","last_deep_scrub_stamp":"2024-09-15T00:11:51.795493+0000","last_clean_scrub_stamp":"2024-09-15T00:11:51.795493+0000","objects_scrubbed":1,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:42:40.398643+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001206129,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12081,"num_read_kb":64593,"num_write":535,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":3188289,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"161'500","reported_seq":5544,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:50.603008+0000","last_change":"2024-09-15T00:11:50.603082+0000","last_active":"2024-09-15T00:11:50.603008+0000","last_peered":"2024-09-15T00:11:50.603008+0000","last_clean":"2024-09-15T00:11:50.603008+0000","last_became_active":"2024-09-15T00:10:58.347628+0000","last_became_peered":"2024-09-15T00:10:58.347628+0000","last_unstale":"2024-09-15T00:11:50.603008+0000","last_undegraded":"2024-09-15T00:11:50.603008+0000","last_fullsized":"2024-09-15T00:11:50.603008+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'500","last_scrub_stamp":"2024-09-15T00:11:50.602891+0000","last_deep_scrub":"161'500","last_deep_scrub_stamp":"2024-09-15T00:11:50.602891+0000","last_clean_scrub_stamp":"2024-09-15T00:11:50.602891+0000","objects_scrubbed":4,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:56:54.435375+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0014193910000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6067,"num_read_kb":40333,"num_write":376,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"161'827","reported_seq":5721,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:54.193852+0000","last_change":"2024-09-15T00:11:54.193882+0000","last_active":"2024-09-15T00:11:54.193852+0000","last_peered":"2024-09-15T00:11:54.193852+0000","last_clean":"2024-09-15T00:11:54.193852+0000","last_became_active":"2024-09-15T00:09:28.895645+0000","last_became_peered":"2024-09-15T00:09:28.895645+0000","last_unstale":"2024-09-15T00:11:54.193852+0000","last_undegraded":"2024-09-15T00:11:54.193852+0000","last_fullsized":"2024-09-15T00:11:54.193852+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'827","last_scrub_stamp":"2024-09-15T00:11:54.193765+0000","last_deep_scrub":"161'827","last_deep_scrub_stamp":"2024-09-15T00:11:54.193765+0000","last_clean_scrub_stamp":"2024-09-15T00:11:54.193765+0000","objects_scrubbed":1,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:16:17.575380+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0019789569999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8748,"num_read_kb":37428,"num_write":973,"num_write_kb":5249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":239143,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'515","reported_seq":6143,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:55.200977+0000","last_change":"2024-09-15T00:11:55.201018+0000","last_active":"2024-09-15T00:11:55.200977+0000","last_peered":"2024-09-15T00:11:55.200977+0000","last_clean":"2024-09-15T00:11:55.200977+0000","last_became_active":"2024-09-15T00:10:58.347433+0000","last_became_peered":"2024-09-15T00:10:58.347433+0000","last_unstale":"2024-09-15T00:11:55.200977+0000","last_undegraded":"2024-09-15T00:11:55.200977+0000","last_fullsized":"2024-09-15T00:11:55.200977+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'515","last_scrub_stamp":"2024-09-15T00:11:55.200864+0000","last_deep_scrub":"161'515","last_deep_scrub_stamp":"2024-09-15T00:11:55.200864+0000","last_clean_scrub_stamp":"2024-09-15T00:11:55.200864+0000","objects_scrubbed":3,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:53:01.333480+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0016878240000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6371,"num_read_kb":39648,"num_write":614,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"137'519","reported_seq":10500,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:50.830636+0000","last_change":"2024-09-15T00:11:50.830666+0000","last_active":"2024-09-15T00:11:50.830636+0000","last_peered":"2024-09-15T00:11:50.830636+0000","last_clean":"2024-09-15T00:11:50.830636+0000","last_became_active":"2024-09-15T00:11:27.814209+0000","last_became_peered":"2024-09-15T00:11:27.814209+0000","last_unstale":"2024-09-15T00:11:50.830636+0000","last_undegraded":"2024-09-15T00:11:50.830636+0000","last_fullsized":"2024-09-15T00:11:50.830636+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'519","last_scrub_stamp":"2024-09-15T00:11:50.830540+0000","last_deep_scrub":"137'519","last_deep_scrub_stamp":"2024-09-15T00:11:50.830540+0000","last_clean_scrub_stamp":"2024-09-15T00:11:50.830540+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:05.507114+0000","scrub_duration":1,"objects_trimmed":1,"snaptrim_duration":0.016246954000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11088,"num_read_kb":77879,"num_write":574,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":10986127,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"161'536","reported_seq":11135,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:49.602027+0000","last_change":"2024-09-15T00:11:49.602077+0000","last_active":"2024-09-15T00:11:49.602027+0000","last_peered":"2024-09-15T00:11:49.602027+0000","last_clean":"2024-09-15T00:11:49.602027+0000","last_became_active":"2024-09-15T00:10:58.346770+0000","last_became_peered":"2024-09-15T00:10:58.346770+0000","last_unstale":"2024-09-15T00:11:49.602027+0000","last_undegraded":"2024-09-15T00:11:49.602027+0000","last_fullsized":"2024-09-15T00:11:49.602027+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'536","last_scrub_stamp":"2024-09-15T00:11:49.601914+0000","last_deep_scrub":"161'536","last_deep_scrub_stamp":"2024-09-15T00:11:49.601914+0000","last_clean_scrub_stamp":"2024-09-15T00:11:49.601914+0000","objects_scrubbed":2,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:35:08.485092+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0015136839999999999,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":13910,"num_read_kb":40588,"num_write":444,"num_write_kb":8754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"161'711","reported_seq":10808,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:51.532867+0000","last_change":"2024-09-15T00:11:51.532912+0000","last_active":"2024-09-15T00:11:51.532867+0000","last_peered":"2024-09-15T00:11:51.532867+0000","last_clean":"2024-09-15T00:11:51.532867+0000","last_became_active":"2024-09-15T00:10:58.350007+0000","last_became_peered":"2024-09-15T00:10:58.350007+0000","last_unstale":"2024-09-15T00:11:51.532867+0000","last_undegraded":"2024-09-15T00:11:51.532867+0000","last_fullsized":"2024-09-15T00:11:51.532867+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'711","last_scrub_stamp":"2024-09-15T00:11:51.532778+0000","last_deep_scrub":"161'711","last_deep_scrub_stamp":"2024-09-15T00:11:51.532778+0000","last_clean_scrub_stamp":"2024-09-15T00:11:51.532778+0000","objects_scrubbed":2,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:59:18.832795+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0019477310000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13597,"num_read_kb":55399,"num_write":535,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'659","reported_seq":8940,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:51.590300+0000","last_change":"2024-09-15T00:11:51.590345+0000","last_active":"2024-09-15T00:11:51.590300+0000","last_peered":"2024-09-15T00:11:51.590300+0000","last_clean":"2024-09-15T00:11:51.590300+0000","last_became_active":"2024-09-15T00:11:28.813725+0000","last_became_peered":"2024-09-15T00:11:28.813725+0000","last_unstale":"2024-09-15T00:11:51.590300+0000","last_undegraded":"2024-09-15T00:11:51.590300+0000","last_fullsized":"2024-09-15T00:11:51.590300+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'659","last_scrub_stamp":"2024-09-15T00:11:51.590205+0000","last_deep_scrub":"161'659","last_deep_scrub_stamp":"2024-09-15T00:11:51.590205+0000","last_clean_scrub_stamp":"2024-09-15T00:11:51.590205+0000","objects_scrubbed":4,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:17:06.613264+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0012280349999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9415,"num_read_kb":64591,"num_write":699,"num_write_kb":14239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":6712878,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"161'863","reported_seq":15060,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:48.756477+0000","last_change":"2024-09-15T00:11:48.756509+0000","last_active":"2024-09-15T00:11:48.756477+0000","last_peered":"2024-09-15T00:11:48.756477+0000","last_clean":"2024-09-15T00:11:48.756477+0000","last_became_active":"2024-09-15T00:11:27.810983+0000","last_became_peered":"2024-09-15T00:11:27.810983+0000","last_unstale":"2024-09-15T00:11:48.756477+0000","last_undegraded":"2024-09-15T00:11:48.756477+0000","last_fullsized":"2024-09-15T00:11:48.756477+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'863","last_scrub_stamp":"2024-09-15T00:11:48.756389+0000","last_deep_scrub":"161'863","last_deep_scrub_stamp":"2024-09-15T00:11:48.756389+0000","last_clean_scrub_stamp":"2024-09-15T00:11:48.756389+0000","objects_scrubbed":4,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:58.361263+0000","scrub_duration":4,"objects_trimmed":2,"snaptrim_duration":0.0050753860000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":16900,"num_read_kb":72084,"num_write":902,"num_write_kb":44641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":14593098,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"161'833","reported_seq":8189,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:53.223200+0000","last_change":"2024-09-15T00:11:53.223229+0000","last_active":"2024-09-15T00:11:53.223200+0000","last_peered":"2024-09-15T00:11:53.223200+0000","last_clean":"2024-09-15T00:11:53.223200+0000","last_became_active":"2024-09-15T00:08:59.946322+0000","last_became_peered":"2024-09-15T00:08:59.946322+0000","last_unstale":"2024-09-15T00:11:53.223200+0000","last_undegraded":"2024-09-15T00:11:53.223200+0000","last_fullsized":"2024-09-15T00:11:53.223200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'833","last_scrub_stamp":"2024-09-15T00:11:53.223100+0000","last_deep_scrub":"161'833","last_deep_scrub_stamp":"2024-09-15T00:11:53.223100+0000","last_clean_scrub_stamp":"2024-09-15T00:11:53.223100+0000","objects_scrubbed":1,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:49:12.708822+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00098720300000000008,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10216,"num_read_kb":33960,"num_write":980,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"161'463","reported_seq":7078,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:57.315080+0000","last_change":"2024-09-15T00:11:57.315148+0000","last_active":"2024-09-15T00:11:57.315080+0000","last_peered":"2024-09-15T00:11:57.315080+0000","last_clean":"2024-09-15T00:11:57.315080+0000","last_became_active":"2024-09-15T00:11:27.810534+0000","last_became_peered":"2024-09-15T00:11:27.810534+0000","last_unstale":"2024-09-15T00:11:57.315080+0000","last_undegraded":"2024-09-15T00:11:57.315080+0000","last_fullsized":"2024-09-15T00:11:57.315080+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'463","last_scrub_stamp":"2024-09-15T00:11:57.314963+0000","last_deep_scrub":"161'463","last_deep_scrub_stamp":"2024-09-15T00:11:57.314963+0000","last_clean_scrub_stamp":"2024-09-15T00:11:57.314963+0000","objects_scrubbed":3,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:01:34.187400+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00081705799999999996,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6309,"num_read_kb":62113,"num_write":514,"num_write_kb":29400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9886793,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"161'578","reported_seq":8469,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:53.354254+0000","last_change":"2024-09-15T00:11:53.354287+0000","last_active":"2024-09-15T00:11:53.354254+0000","last_peered":"2024-09-15T00:11:53.354254+0000","last_clean":"2024-09-15T00:11:53.354254+0000","last_became_active":"2024-09-15T00:10:58.353801+0000","last_became_peered":"2024-09-15T00:10:58.353801+0000","last_unstale":"2024-09-15T00:11:53.354254+0000","last_undegraded":"2024-09-15T00:11:53.354254+0000","last_fullsized":"2024-09-15T00:11:53.354254+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'578","last_scrub_stamp":"2024-09-15T00:11:53.354168+0000","last_deep_scrub":"161'578","last_deep_scrub_stamp":"2024-09-15T00:11:53.354168+0000","last_clean_scrub_stamp":"2024-09-15T00:11:53.354168+0000","objects_scrubbed":1,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:22:02.058025+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.001503938,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9435,"num_read_kb":45249,"num_write":397,"num_write_kb":12856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":319,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:51.331189+0000","last_change":"2024-09-15T00:11:51.331213+0000","last_active":"2024-09-15T00:11:51.331189+0000","last_peered":"2024-09-15T00:11:51.331189+0000","last_clean":"2024-09-15T00:11:51.331189+0000","last_became_active":"2024-09-15T00:10:58.352314+0000","last_became_peered":"2024-09-15T00:10:58.352314+0000","last_unstale":"2024-09-15T00:11:51.331189+0000","last_undegraded":"2024-09-15T00:11:51.331189+0000","last_fullsized":"2024-09-15T00:11:51.331189+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:51.331098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:51.331098+0000","last_clean_scrub_stamp":"2024-09-15T00:11:51.331098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:16:19.974915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047811099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"161'459","reported_seq":3581,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:46.798463+0000","last_change":"2024-09-15T00:11:46.798515+0000","last_active":"2024-09-15T00:11:46.798463+0000","last_peered":"2024-09-15T00:11:46.798463+0000","last_clean":"2024-09-15T00:11:46.798463+0000","last_became_active":"2024-09-15T00:11:26.810222+0000","last_became_peered":"2024-09-15T00:11:26.810222+0000","last_unstale":"2024-09-15T00:11:46.798463+0000","last_undegraded":"2024-09-15T00:11:46.798463+0000","last_fullsized":"2024-09-15T00:11:46.798463+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'459","last_scrub_stamp":"2024-09-15T00:11:46.798335+0000","last_deep_scrub":"161'459","last_deep_scrub_stamp":"2024-09-15T00:11:46.798335+0000","last_clean_scrub_stamp":"2024-09-15T00:11:46.798335+0000","objects_scrubbed":4,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:53:00.877802+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00099603400000000003,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2834,"num_read_kb":21154,"num_write":404,"num_write_kb":9106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":6729262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":311,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:47.481287+0000","last_change":"2024-09-15T00:11:47.481307+0000","last_active":"2024-09-15T00:11:47.481287+0000","last_peered":"2024-09-15T00:11:47.481287+0000","last_clean":"2024-09-15T00:11:47.481287+0000","last_became_active":"2024-09-15T00:08:50.830004+0000","last_became_peered":"2024-09-15T00:08:50.830004+0000","last_unstale":"2024-09-15T00:11:47.481287+0000","last_undegraded":"2024-09-15T00:11:47.481287+0000","last_fullsized":"2024-09-15T00:11:47.481287+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:47.481210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:47.481210+0000","last_clean_scrub_stamp":"2024-09-15T00:11:47.481210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:10.788853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056544500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"161'302","reported_seq":3614,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:49.563661+0000","last_change":"2024-09-15T00:11:49.563687+0000","last_active":"2024-09-15T00:11:49.563661+0000","last_peered":"2024-09-15T00:11:49.563661+0000","last_clean":"2024-09-15T00:11:49.563661+0000","last_became_active":"2024-09-15T00:08:59.947651+0000","last_became_peered":"2024-09-15T00:08:59.947651+0000","last_unstale":"2024-09-15T00:11:49.563661+0000","last_undegraded":"2024-09-15T00:11:49.563661+0000","last_fullsized":"2024-09-15T00:11:49.563661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'302","last_scrub_stamp":"2024-09-15T00:11:49.563587+0000","last_deep_scrub":"161'302","last_deep_scrub_stamp":"2024-09-15T00:11:49.563587+0000","last_clean_scrub_stamp":"2024-09-15T00:11:49.563587+0000","objects_scrubbed":1,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:33:55.877936+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.001294859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3137,"num_read_kb":36813,"num_write":322,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":331,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:55.346731+0000","last_change":"2024-09-15T00:11:55.346754+0000","last_active":"2024-09-15T00:11:55.346731+0000","last_peered":"2024-09-15T00:11:55.346731+0000","last_clean":"2024-09-15T00:11:55.346731+0000","last_became_active":"2024-09-15T00:11:23.820814+0000","last_became_peered":"2024-09-15T00:11:23.820814+0000","last_unstale":"2024-09-15T00:11:55.346731+0000","last_undegraded":"2024-09-15T00:11:55.346731+0000","last_fullsized":"2024-09-15T00:11:55.346731+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:55.346644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:55.346644+0000","last_clean_scrub_stamp":"2024-09-15T00:11:55.346644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:58:48.733669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066667300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"161'808","reported_seq":7280,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:48.508563+0000","last_change":"2024-09-15T00:11:48.508597+0000","last_active":"2024-09-15T00:11:48.508563+0000","last_peered":"2024-09-15T00:11:48.508563+0000","last_clean":"2024-09-15T00:11:48.508563+0000","last_became_active":"2024-09-15T00:08:59.942341+0000","last_became_peered":"2024-09-15T00:08:59.942341+0000","last_unstale":"2024-09-15T00:11:48.508563+0000","last_undegraded":"2024-09-15T00:11:48.508563+0000","last_fullsized":"2024-09-15T00:11:48.508563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'808","last_scrub_stamp":"2024-09-15T00:11:48.508481+0000","last_deep_scrub":"161'808","last_deep_scrub_stamp":"2024-09-15T00:11:48.508481+0000","last_clean_scrub_stamp":"2024-09-15T00:11:48.508481+0000","objects_scrubbed":4,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:42.885617+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0015707939999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9943,"num_read_kb":67757,"num_write":840,"num_write_kb":50219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":330,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:52.575511+0000","last_change":"2024-09-15T00:11:52.575537+0000","last_active":"2024-09-15T00:11:52.575511+0000","last_peered":"2024-09-15T00:11:52.575511+0000","last_clean":"2024-09-15T00:11:52.575511+0000","last_became_active":"2024-09-15T00:11:23.823594+0000","last_became_peered":"2024-09-15T00:11:23.823594+0000","last_unstale":"2024-09-15T00:11:52.575511+0000","last_undegraded":"2024-09-15T00:11:52.575511+0000","last_fullsized":"2024-09-15T00:11:52.575511+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:52.575416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:52.575416+0000","last_clean_scrub_stamp":"2024-09-15T00:11:52.575416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:41:37.257560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056638200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"161'603","reported_seq":6000,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:52.205470+0000","last_change":"2024-09-15T00:11:52.205508+0000","last_active":"2024-09-15T00:11:52.205470+0000","last_peered":"2024-09-15T00:11:52.205470+0000","last_clean":"2024-09-15T00:11:52.205470+0000","last_became_active":"2024-09-15T00:08:59.947109+0000","last_became_peered":"2024-09-15T00:08:59.947109+0000","last_unstale":"2024-09-15T00:11:52.205470+0000","last_undegraded":"2024-09-15T00:11:52.205470+0000","last_fullsized":"2024-09-15T00:11:52.205470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'603","last_scrub_stamp":"2024-09-15T00:11:52.205384+0000","last_deep_scrub":"161'603","last_deep_scrub_stamp":"2024-09-15T00:11:52.205384+0000","last_clean_scrub_stamp":"2024-09-15T00:11:52.205384+0000","objects_scrubbed":5,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:42:51.394094+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0022079199999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6563,"num_read_kb":34828,"num_write":657,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"161'673","reported_seq":6120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:50.578701+0000","last_change":"2024-09-15T00:11:50.578757+0000","last_active":"2024-09-15T00:11:50.578701+0000","last_peered":"2024-09-15T00:11:50.578701+0000","last_clean":"2024-09-15T00:11:50.578701+0000","last_became_active":"2024-09-15T00:11:26.812061+0000","last_became_peered":"2024-09-15T00:11:26.812061+0000","last_unstale":"2024-09-15T00:11:50.578701+0000","last_undegraded":"2024-09-15T00:11:50.578701+0000","last_fullsized":"2024-09-15T00:11:50.578701+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'673","last_scrub_stamp":"2024-09-15T00:11:50.578579+0000","last_deep_scrub":"161'673","last_deep_scrub_stamp":"2024-09-15T00:11:50.578579+0000","last_clean_scrub_stamp":"2024-09-15T00:11:50.578579+0000","objects_scrubbed":4,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:12:33.577256+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0013113269999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7232,"num_read_kb":26564,"num_write":565,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"161'3","reported_seq":316,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:50.200715+0000","last_change":"2024-09-15T00:11:50.200762+0000","last_active":"2024-09-15T00:11:50.200715+0000","last_peered":"2024-09-15T00:11:50.200715+0000","last_clean":"2024-09-15T00:11:50.200715+0000","last_became_active":"2024-09-15T00:08:50.833179+0000","last_became_peered":"2024-09-15T00:08:50.833179+0000","last_unstale":"2024-09-15T00:11:50.200715+0000","last_undegraded":"2024-09-15T00:11:50.200715+0000","last_fullsized":"2024-09-15T00:11:50.200715+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'3","last_scrub_stamp":"2024-09-15T00:11:50.200620+0000","last_deep_scrub":"161'3","last_deep_scrub_stamp":"2024-09-15T00:11:50.200620+0000","last_clean_scrub_stamp":"2024-09-15T00:11:50.200620+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:34:49.268958+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00098024599999999994,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"161'715","reported_seq":7169,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:49.497338+0000","last_change":"2024-09-15T00:11:49.497362+0000","last_active":"2024-09-15T00:11:49.497338+0000","last_peered":"2024-09-15T00:11:49.497338+0000","last_clean":"2024-09-15T00:11:49.497338+0000","last_became_active":"2024-09-15T00:08:59.943137+0000","last_became_peered":"2024-09-15T00:08:59.943137+0000","last_unstale":"2024-09-15T00:11:49.497338+0000","last_undegraded":"2024-09-15T00:11:49.497338+0000","last_fullsized":"2024-09-15T00:11:49.497338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'715","last_scrub_stamp":"2024-09-15T00:11:49.497261+0000","last_deep_scrub":"161'715","last_deep_scrub_stamp":"2024-09-15T00:11:49.497261+0000","last_clean_scrub_stamp":"2024-09-15T00:11:49.497261+0000","objects_scrubbed":1,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:01:36.647730+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001570948,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7803,"num_read_kb":54109,"num_write":924,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":315,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:47.532626+0000","last_change":"2024-09-15T00:11:47.532650+0000","last_active":"2024-09-15T00:11:47.532626+0000","last_peered":"2024-09-15T00:11:47.532626+0000","last_clean":"2024-09-15T00:11:47.532626+0000","last_became_active":"2024-09-15T00:08:50.828150+0000","last_became_peered":"2024-09-15T00:08:50.828150+0000","last_unstale":"2024-09-15T00:11:47.532626+0000","last_undegraded":"2024-09-15T00:11:47.532626+0000","last_fullsized":"2024-09-15T00:11:47.532626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:47.532536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:47.532536+0000","last_clean_scrub_stamp":"2024-09-15T00:11:47.532536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:42:56.028867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048917999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"161'632","reported_seq":3728,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:47.798105+0000","last_change":"2024-09-15T00:11:47.798134+0000","last_active":"2024-09-15T00:11:47.798105+0000","last_peered":"2024-09-15T00:11:47.798105+0000","last_clean":"2024-09-15T00:11:47.798105+0000","last_became_active":"2024-09-15T00:11:26.813953+0000","last_became_peered":"2024-09-15T00:11:26.813953+0000","last_unstale":"2024-09-15T00:11:47.798105+0000","last_undegraded":"2024-09-15T00:11:47.798105+0000","last_fullsized":"2024-09-15T00:11:47.798105+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'632","last_scrub_stamp":"2024-09-15T00:11:47.798033+0000","last_deep_scrub":"161'632","last_deep_scrub_stamp":"2024-09-15T00:11:47.798033+0000","last_clean_scrub_stamp":"2024-09-15T00:11:47.798033+0000","objects_scrubbed":5,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:41.096617+0000","scrub_duration":4,"objects_trimmed":2,"snaptrim_duration":0.00071073200000000005,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5290,"num_read_kb":27132,"num_write":878,"num_write_kb":8037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2365199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":312,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:48.410297+0000","last_change":"2024-09-15T00:11:48.410327+0000","last_active":"2024-09-15T00:11:48.410297+0000","last_peered":"2024-09-15T00:11:48.410297+0000","last_clean":"2024-09-15T00:11:48.410297+0000","last_became_active":"2024-09-15T00:08:50.826966+0000","last_became_peered":"2024-09-15T00:08:50.826966+0000","last_unstale":"2024-09-15T00:11:48.410297+0000","last_undegraded":"2024-09-15T00:11:48.410297+0000","last_fullsized":"2024-09-15T00:11:48.410297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T00:11:48.410199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T00:11:48.410199+0000","last_clean_scrub_stamp":"2024-09-15T00:11:48.410199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:41:00.138142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045469100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"161'497","reported_seq":6893,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:50.524057+0000","last_change":"2024-09-15T00:11:50.524093+0000","last_active":"2024-09-15T00:11:50.524057+0000","last_peered":"2024-09-15T00:11:50.524057+0000","last_clean":"2024-09-15T00:11:50.524057+0000","last_became_active":"2024-09-15T00:08:59.945201+0000","last_became_peered":"2024-09-15T00:08:59.945201+0000","last_unstale":"2024-09-15T00:11:50.524057+0000","last_undegraded":"2024-09-15T00:11:50.524057+0000","last_fullsized":"2024-09-15T00:11:50.524057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'497","last_scrub_stamp":"2024-09-15T00:11:50.523965+0000","last_deep_scrub":"161'497","last_deep_scrub_stamp":"2024-09-15T00:11:50.523965+0000","last_clean_scrub_stamp":"2024-09-15T00:11:50.523965+0000","objects_scrubbed":4,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:41:03.994949+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001215973,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6232,"num_read_kb":25476,"num_write":407,"num_write_kb":5284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"161'2","reported_seq":315,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:49.232959+0000","last_change":"2024-09-15T00:11:49.232991+0000","last_active":"2024-09-15T00:11:49.232959+0000","last_peered":"2024-09-15T00:11:49.232959+0000","last_clean":"2024-09-15T00:11:49.232959+0000","last_became_active":"2024-09-15T00:08:50.829760+0000","last_became_peered":"2024-09-15T00:08:50.829760+0000","last_unstale":"2024-09-15T00:11:49.232959+0000","last_undegraded":"2024-09-15T00:11:49.232959+0000","last_fullsized":"2024-09-15T00:11:49.232959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'2","last_scrub_stamp":"2024-09-15T00:11:49.232852+0000","last_deep_scrub":"161'2","last_deep_scrub_stamp":"2024-09-15T00:11:49.232852+0000","last_clean_scrub_stamp":"2024-09-15T00:11:49.232852+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:20:52.771944+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00047659800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"161'119","reported_seq":493,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:56.302977+0000","last_change":"2024-09-15T00:11:56.303019+0000","last_active":"2024-09-15T00:11:56.302977+0000","last_peered":"2024-09-15T00:11:56.302977+0000","last_clean":"2024-09-15T00:11:56.302977+0000","last_became_active":"2024-09-15T00:11:26.809081+0000","last_became_peered":"2024-09-15T00:11:26.809081+0000","last_unstale":"2024-09-15T00:11:56.302977+0000","last_undegraded":"2024-09-15T00:11:56.302977+0000","last_fullsized":"2024-09-15T00:11:56.302977+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'119","last_scrub_stamp":"2024-09-15T00:11:56.302877+0000","last_deep_scrub":"161'119","last_deep_scrub_stamp":"2024-09-15T00:11:56.302877+0000","last_clean_scrub_stamp":"2024-09-15T00:11:56.302877+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:13.568749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"161'572","reported_seq":11127,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:48.586705+0000","last_change":"2024-09-15T00:11:48.586741+0000","last_active":"2024-09-15T00:11:48.586705+0000","last_peered":"2024-09-15T00:11:48.586705+0000","last_clean":"2024-09-15T00:11:48.586705+0000","last_became_active":"2024-09-15T00:08:59.945429+0000","last_became_peered":"2024-09-15T00:08:59.945429+0000","last_unstale":"2024-09-15T00:11:48.586705+0000","last_undegraded":"2024-09-15T00:11:48.586705+0000","last_fullsized":"2024-09-15T00:11:48.586705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'572","last_scrub_stamp":"2024-09-15T00:11:48.586627+0000","last_deep_scrub":"161'572","last_deep_scrub_stamp":"2024-09-15T00:11:48.586627+0000","last_clean_scrub_stamp":"2024-09-15T00:11:48.586627+0000","objects_scrubbed":2,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:23.478239+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00098862600000000004,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":12740,"num_read_kb":88117,"num_write":607,"num_write_kb":11282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"161'512","reported_seq":9043,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:48.644484+0000","last_change":"2024-09-15T00:11:48.644540+0000","last_active":"2024-09-15T00:11:48.644484+0000","last_peered":"2024-09-15T00:11:48.644484+0000","last_clean":"2024-09-15T00:11:48.644484+0000","last_became_active":"2024-09-15T00:10:58.346994+0000","last_became_peered":"2024-09-15T00:10:58.346994+0000","last_unstale":"2024-09-15T00:11:48.644484+0000","last_undegraded":"2024-09-15T00:11:48.644484+0000","last_fullsized":"2024-09-15T00:11:48.644484+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'512","last_scrub_stamp":"2024-09-15T00:11:48.644381+0000","last_deep_scrub":"161'512","last_deep_scrub_stamp":"2024-09-15T00:11:48.644381+0000","last_clean_scrub_stamp":"2024-09-15T00:11:48.644381+0000","objects_scrubbed":1,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:06:46.716396+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00088599499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":9287,"num_read_kb":49680,"num_write":419,"num_write_kb":13143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"161'351","reported_seq":2548,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:52.346286+0000","last_change":"2024-09-15T00:11:52.346323+0000","last_active":"2024-09-15T00:11:52.346286+0000","last_peered":"2024-09-15T00:11:52.346286+0000","last_clean":"2024-09-15T00:11:52.346286+0000","last_became_active":"2024-09-15T00:10:58.353160+0000","last_became_peered":"2024-09-15T00:10:58.353160+0000","last_unstale":"2024-09-15T00:11:52.346286+0000","last_undegraded":"2024-09-15T00:11:52.346286+0000","last_fullsized":"2024-09-15T00:11:52.346286+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'351","last_scrub_stamp":"2024-09-15T00:11:52.346196+0000","last_deep_scrub":"161'351","last_deep_scrub_stamp":"2024-09-15T00:11:52.346196+0000","last_clean_scrub_stamp":"2024-09-15T00:11:52.346196+0000","objects_scrubbed":5,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:44:59.996121+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.001573252,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":2498,"num_read_kb":16284,"num_write":316,"num_write_kb":1926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"161'346","reported_seq":4097,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:53.573225+0000","last_change":"2024-09-15T00:11:53.573319+0000","last_active":"2024-09-15T00:11:53.573225+0000","last_peered":"2024-09-15T00:11:53.573225+0000","last_clean":"2024-09-15T00:11:53.573225+0000","last_became_active":"2024-09-15T00:11:27.812092+0000","last_became_peered":"2024-09-15T00:11:27.812092+0000","last_unstale":"2024-09-15T00:11:53.573225+0000","last_undegraded":"2024-09-15T00:11:53.573225+0000","last_fullsized":"2024-09-15T00:11:53.573225+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'346","last_scrub_stamp":"2024-09-15T00:11:53.573131+0000","last_deep_scrub":"161'346","last_deep_scrub_stamp":"2024-09-15T00:11:53.573131+0000","last_clean_scrub_stamp":"2024-09-15T00:11:53.573131+0000","objects_scrubbed":4,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:37.370941+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.000721333,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":3470,"num_read_kb":33354,"num_write":301,"num_write_kb":6941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":4385400,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"161'1091","reported_seq":6166,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:58.207103+0000","last_change":"2024-09-15T00:11:58.207136+0000","last_active":"2024-09-15T00:11:58.207103+0000","last_peered":"2024-09-15T00:11:58.207103+0000","last_clean":"2024-09-15T00:11:58.207103+0000","last_became_active":"2024-09-15T00:11:27.811047+0000","last_became_peered":"2024-09-15T00:11:27.811047+0000","last_unstale":"2024-09-15T00:11:58.207103+0000","last_undegraded":"2024-09-15T00:11:58.207103+0000","last_fullsized":"2024-09-15T00:11:58.207103+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'1091","last_scrub_stamp":"2024-09-15T00:11:58.207012+0000","last_deep_scrub":"161'1091","last_deep_scrub_stamp":"2024-09-15T00:11:58.207012+0000","last_clean_scrub_stamp":"2024-09-15T00:11:58.207012+0000","objects_scrubbed":2,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:27:28.673405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0021428929999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8751,"num_read_kb":31066,"num_write":1650,"num_write_kb":11591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":2260644,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"161'477","reported_seq":5198,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:49.407485+0000","last_change":"2024-09-15T00:11:49.407524+0000","last_active":"2024-09-15T00:11:49.407485+0000","last_peered":"2024-09-15T00:11:49.407485+0000","last_clean":"2024-09-15T00:11:49.407485+0000","last_became_active":"2024-09-15T00:08:59.944315+0000","last_became_peered":"2024-09-15T00:08:59.944315+0000","last_unstale":"2024-09-15T00:11:49.407485+0000","last_undegraded":"2024-09-15T00:11:49.407485+0000","last_fullsized":"2024-09-15T00:11:49.407485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'477","last_scrub_stamp":"2024-09-15T00:11:49.407402+0000","last_deep_scrub":"161'477","last_deep_scrub_stamp":"2024-09-15T00:11:49.407402+0000","last_clean_scrub_stamp":"2024-09-15T00:11:49.407402+0000","objects_scrubbed":5,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:21.001943+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0012186440000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6281,"num_read_kb":25421,"num_write":438,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"161'323","reported_seq":4504,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:56.247245+0000","last_change":"2024-09-15T00:11:56.247280+0000","last_active":"2024-09-15T00:11:56.247245+0000","last_peered":"2024-09-15T00:11:56.247245+0000","last_clean":"2024-09-15T00:11:56.247245+0000","last_became_active":"2024-09-15T00:10:58.347480+0000","last_became_peered":"2024-09-15T00:10:58.347480+0000","last_unstale":"2024-09-15T00:11:56.247245+0000","last_undegraded":"2024-09-15T00:11:56.247245+0000","last_fullsized":"2024-09-15T00:11:56.247245+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'323","last_scrub_stamp":"2024-09-15T00:11:56.247127+0000","last_deep_scrub":"161'323","last_deep_scrub_stamp":"2024-09-15T00:11:56.247127+0000","last_clean_scrub_stamp":"2024-09-15T00:11:56.247127+0000","objects_scrubbed":2,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:02.995887+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0015201839999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3806,"num_read_kb":22330,"num_write":293,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"161'778","reported_seq":5232,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:51.196960+0000","last_change":"2024-09-15T00:11:51.197005+0000","last_active":"2024-09-15T00:11:51.196960+0000","last_peered":"2024-09-15T00:11:51.196960+0000","last_clean":"2024-09-15T00:11:51.196960+0000","last_became_active":"2024-09-15T00:08:59.946259+0000","last_became_peered":"2024-09-15T00:08:59.946259+0000","last_unstale":"2024-09-15T00:11:51.196960+0000","last_undegraded":"2024-09-15T00:11:51.196960+0000","last_fullsized":"2024-09-15T00:11:51.196960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'778","last_scrub_stamp":"2024-09-15T00:11:51.196845+0000","last_deep_scrub":"161'778","last_deep_scrub_stamp":"2024-09-15T00:11:51.196845+0000","last_clean_scrub_stamp":"2024-09-15T00:11:51.196845+0000","objects_scrubbed":1,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:26:21.052698+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0018611719999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7063,"num_read_kb":22283,"num_write":819,"num_write_kb":6499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"161'347","reported_seq":6258,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:57.216154+0000","last_change":"2024-09-15T00:11:57.216187+0000","last_active":"2024-09-15T00:11:57.216154+0000","last_peered":"2024-09-15T00:11:57.216154+0000","last_clean":"2024-09-15T00:11:57.216154+0000","last_became_active":"2024-09-15T00:11:26.809733+0000","last_became_peered":"2024-09-15T00:11:26.809733+0000","last_unstale":"2024-09-15T00:11:57.216154+0000","last_undegraded":"2024-09-15T00:11:57.216154+0000","last_fullsized":"2024-09-15T00:11:57.216154+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'347","last_scrub_stamp":"2024-09-15T00:11:57.216043+0000","last_deep_scrub":"161'347","last_deep_scrub_stamp":"2024-09-15T00:11:57.216043+0000","last_clean_scrub_stamp":"2024-09-15T00:11:57.216043+0000","objects_scrubbed":2,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:13:56.223692+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001014647,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5082,"num_read_kb":32663,"num_write":383,"num_write_kb":18238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":7454943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"134'1204","reported_seq":4841,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:54.306370+0000","last_change":"2024-09-15T00:11:54.306416+0000","last_active":"2024-09-15T00:11:54.306370+0000","last_peered":"2024-09-15T00:11:54.306370+0000","last_clean":"2024-09-15T00:11:54.306370+0000","last_became_active":"2024-09-15T00:10:58.354658+0000","last_became_peered":"2024-09-15T00:10:58.354658+0000","last_unstale":"2024-09-15T00:11:54.306370+0000","last_undegraded":"2024-09-15T00:11:54.306370+0000","last_fullsized":"2024-09-15T00:11:54.306370+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'1204","last_scrub_stamp":"2024-09-15T00:11:54.306267+0000","last_deep_scrub":"134'1204","last_deep_scrub_stamp":"2024-09-15T00:11:54.306267+0000","last_clean_scrub_stamp":"2024-09-15T00:11:54.306267+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:17:08.641950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185029,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7974,"num_read_kb":21277,"num_write":1815,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"161'1282","reported_seq":10335,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:49.796744+0000","last_change":"2024-09-15T00:11:49.796772+0000","last_active":"2024-09-15T00:11:49.796744+0000","last_peered":"2024-09-15T00:11:49.796744+0000","last_clean":"2024-09-15T00:11:49.796744+0000","last_became_active":"2024-09-15T00:11:27.813284+0000","last_became_peered":"2024-09-15T00:11:27.813284+0000","last_unstale":"2024-09-15T00:11:49.796744+0000","last_undegraded":"2024-09-15T00:11:49.796744+0000","last_fullsized":"2024-09-15T00:11:49.796744+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'1282","last_scrub_stamp":"2024-09-15T00:11:49.796665+0000","last_deep_scrub":"161'1282","last_deep_scrub_stamp":"2024-09-15T00:11:49.796665+0000","last_clean_scrub_stamp":"2024-09-15T00:11:49.796665+0000","objects_scrubbed":4,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:02:17.890042+0000","scrub_duration":4,"objects_trimmed":2,"snaptrim_duration":0.010772937999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":12900,"num_read_kb":112467,"num_write":1928,"num_write_kb":43546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":22424184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"161'244","reported_seq":2516,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-15T00:11:50.373413+0000","last_change":"2024-09-15T00:11:50.373444+0000","last_active":"2024-09-15T00:11:50.373413+0000","last_peered":"2024-09-15T00:11:50.373413+0000","last_clean":"2024-09-15T00:11:50.373413+0000","last_became_active":"2024-09-15T00:08:59.943778+0000","last_became_peered":"2024-09-15T00:08:59.943778+0000","last_unstale":"2024-09-15T00:11:50.373413+0000","last_undegraded":"2024-09-15T00:11:50.373413+0000","last_fullsized":"2024-09-15T00:11:50.373413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'244","last_scrub_stamp":"2024-09-15T00:11:50.373328+0000","last_deep_scrub":"161'244","last_deep_scrub_stamp":"2024-09-15T00:11:50.373328+0000","last_clean_scrub_stamp":"2024-09-15T00:11:50.373328+0000","objects_scrubbed":1,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:21:46.221129+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0013592509999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1867,"num_read_kb":15504,"num_write":229,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":79,"num_read":254890,"num_read_kb":1398144,"num_write":21738,"num_write_kb":428109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2462,"num_bytes_recovered":92274536,"num_keys_recovered":184,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26303,"internal_metadata":0},"log_size":19756,"ondisk_log_size":19756,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":27652,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640301,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59440,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312400,"statfs":{"total":96636764160,"available":96575897600,"internally_reserved":0,"allocated":1933312,"data_stored":811415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":118,"seq":506806140944,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34288,"kb_used_data":1888,"kb_used_omap":31,"kb_used_meta":32352,"kb_avail":94337552,"statfs":{"total":96636764160,"available":96601653248,"internally_reserved":0,"allocated":1933312,"data_stored":811415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31754,"internal_metadata":33129462},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640302,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81700,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290140,"statfs":{"total":96636764160,"available":96553103360,"internally_reserved":0,"allocated":1986560,"data_stored":1401764,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640301,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59436,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312404,"statfs":{"total":96636764160,"available":96575901696,"internally_reserved":0,"allocated":1929216,"data_stored":811396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640301,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1937408,"data_stored":811434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":133,"seq":571230650381,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39572,"kb_used_data":1924,"kb_used_omap":28,"kb_used_meta":37603,"kb_avail":94332268,"statfs":{"total":96636764160,"available":96596242432,"internally_reserved":0,"allocated":1970176,"data_stored":1401764,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28896,"internal_metadata":38506272},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9562,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8528,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15263,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11040,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:12:09.373 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T00:12:09.676 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T00:12:09.677 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T00:12:09.677 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T00:12:09.677 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:09.715 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T00:12:09.715 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T00:12:09.715 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:09.785 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T00:12:09.785 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T00:12:09.786 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:09.852 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T00:12:09.852 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T00:12:09.852 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:09.891 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T00:12:09.892 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T00:12:09.892 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:09.966 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T00:12:09.966 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T00:12:09.966 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:10.041 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T00:12:10.042 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T00:12:10.042 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T00:12:10.042 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:10.110 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T00:12:10.111 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T00:12:10.111 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T00:12:10.111 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:10.122 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T00:12:10.122 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T00:12:10.122 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:10.173 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T00:12:10.174 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T00:12:10.174 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:12:10.186 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T00:12:10.186 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T00:12:10.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-15T00:12:10.247 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:12:10.248 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T00:12:10.446 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:12:10.446 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T00:12:10.519 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:12:10.519 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T00:12:10.570 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-15T00:12:10.570 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T00:12:10.788 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-15T00:12:10.789 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T00:12:10.872 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-15T00:12:10.872 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T00:12:10.928 INFO:tasks.ceph:Archiving mon data... 2024-09-15T00:12:10.929 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/data/mon.a.tgz 2024-09-15T00:12:10.929 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-15T00:12:10.945 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.08k3TrpgDz 2024-09-15T00:12:10.946 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.08k3TrpgDz 2024-09-15T00:12:11.092 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.08k3TrpgDz 2024-09-15T00:12:11.177 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.08k3TrpgDz is 321KB 2024-09-15T00:12:11.274 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.08k3TrpgDz 2024-09-15T00:12:11.290 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/data/mon.c.tgz 2024-09-15T00:12:11.290 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-15T00:12:11.345 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.gOd1Mw0sTG 2024-09-15T00:12:11.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.gOd1Mw0sTG 2024-09-15T00:12:11.504 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.gOd1Mw0sTG 2024-09-15T00:12:11.589 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.gOd1Mw0sTG is 438KB 2024-09-15T00:12:11.687 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.gOd1Mw0sTG 2024-09-15T00:12:11.702 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/data/mon.b.tgz 2024-09-15T00:12:11.703 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-15T00:12:11.721 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.aX75m2WIT8 2024-09-15T00:12:11.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.aX75m2WIT8 2024-09-15T00:12:11.897 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.aX75m2WIT8 2024-09-15T00:12:11.981 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.aX75m2WIT8 is 438KB 2024-09-15T00:12:12.077 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.aX75m2WIT8 2024-09-15T00:12:12.092 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T00:12:12.092 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T00:12:12.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T00:12:12.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T00:12:12.181 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T00:12:12.211 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T00:12:12.228 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T00:12:12.228 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T00:12:12.229 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/remote/smithi027/crash 2024-09-15T00:12:12.230 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T00:12:12.255 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/remote/smithi079/crash 2024-09-15T00:12:12.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T00:12:12.281 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/remote/smithi092/crash 2024-09-15T00:12:12.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T00:12:12.411 INFO:tasks.ceph:Compressing logs... 2024-09-15T00:12:12.411 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T00:12:12.414 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T00:12:12.416 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T00:12:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T00:12:12.445 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32139.log 2024-09-15T00:12:12.445 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32099.log 2024-09-15T00:12:12.445 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T00:12:12.445 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32139.log.gz 2024-09-15T00:12:12.445 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32099.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T00:12:12.445 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32099.log.gz 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41786.log 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31963.log 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/objectstore_tool.41786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32019.log 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31963.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31963.log.gz 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32179.log 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32019.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32019.log.gz 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32179.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32179.log.gz 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T00:12:12.446 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32059.log 2024-09-15T00:12:12.447 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.32059.log: 85.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T00:12:12.447 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32059.log.gz 2024-09-15T00:12:12.452 INFO:teuthology.orchestra.run.smithi079.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.41786.log.gz 2024-09-15T00:12:12.489 INFO:teuthology.orchestra.run.smithi079.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T00:12:12.552 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27214.log 2024-09-15T00:12:12.552 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29960.log 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28839.log 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27214.log.gz 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29120.log 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29960.log.gz 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28882.log 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28839.log.gz 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27504.log 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29120.log.gz 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29933.log 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.28882.log.gz 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29689.log 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.27504.log.gz 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.26984.log 2024-09-15T00:12:12.553 INFO:teuthology.orchestra.run.smithi092.stderr: 52.4% -- replaced with /var/log/ceph/ceph-client.admin.29933.log.gz 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27529.log 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.29689.log.gz 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.26984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28098.log 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.26984.log.gz 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27669.log 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27529.log.gz 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29759.log 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr: 76.3% -- replaced with /var/log/ceph/ceph-client.admin.28098.log.gz 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28497.log 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27669.log.gz 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29574.log 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.29759.log.gz 2024-09-15T00:12:12.554 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27388.log 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29574.log: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.28497.log.gz 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27459.log 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29574.log.gz 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28812.log 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27388.log.gz 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29224.log 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27459.log.gz 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29540.log 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.28812.log.gz 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27152.log 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29224.log.gz 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28347.log 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29540.log.gz 2024-09-15T00:12:12.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29504.log 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.27152.log.gz 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28873.log 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28347.log.gz 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29750.log 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29504.log.gz 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28055.log 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28873.log.gz 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27468.log 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29750.log.gz 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27026.log 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28055.log.gz 2024-09-15T00:12:12.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29163.log 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.27468.log.gz 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28522.log 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.27026.log.gz 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29434.log 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.29163.log.gz 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29863.log 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28522.log.gz 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/ceph-client.admin.29434.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28277.log 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29434.log.gz 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28170.log 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.29863.log.gz 2024-09-15T00:12:12.557 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27143.log 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28277.log.gz 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29969.log 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28170.log.gz 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27293.log 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27143.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27143.log.gz 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29294.log 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.29969.log.gz 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28918.log 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.27293.log.gz 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28803.log 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29294.log: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29294.log.gz 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27739.log 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29924.log 2024-09-15T00:12:12.558 INFO:teuthology.orchestra.run.smithi092.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.28918.log.gz 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28803.log.gz 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29269.log 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27739.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27739.log.gz 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29549.log 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29924.log.gz 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28708.log 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28356.log 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.29269.log.gz 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr: 95.9%/var/log/ceph/ceph-client.admin.28708.log: -- replaced with /var/log/ceph/ceph-client.admin.29549.log.gz 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27397.log 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.28708.log.gz 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28179.log 2024-09-15T00:12:12.559 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27397.log: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.28356.log.gz 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 53.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.27397.log.gz 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr: /var/log/ceph/ceph-client.admin.28250.log 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29260.log 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr: 52.2% -- replaced with /var/log/ceph/ceph-client.admin.28179.log.gz 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29154.log 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.28250.log.gz 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28637.log 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29260.log.gz 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28216.log 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29154.log.gz 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27635.log 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.28637.log.gz 2024-09-15T00:12:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28778.log 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.28216.log.gz 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27784.log 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27635.log.gz 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29339.log 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27425.log 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.28778.log.gz 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.27784.log.gz 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28943.log 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.admin.29339.log.gz 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27714.log 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27425.log.gz 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28943.log: gzip 19.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.28943.log.gz --verbose 2024-09-15T00:12:12.561 INFO:teuthology.orchestra.run.smithi092.stderr: -- /var/log/ceph/ceph-client.admin.29014.log 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29050.log 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.27714.log.gz 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27117.log 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29014.log.gz 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27318.log 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29050.log.gz 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/ceph-client.admin.27117.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27915.log 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27318.log: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.27117.log.gz 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27318.log.gz 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29829.log 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27354.log 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27915.log.gz 2024-09-15T00:12:12.562 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29513.log/var/log/ceph/ceph-client.admin.29829.log: 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.29829.log.gz 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27854.log 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27354.log.gz 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27327.log 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.29513.log.gz 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29619.log 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.27854.log.gz 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29723.log 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr: 75.6% -- replaced with /var/log/ceph/ceph-client.admin.27327.log.gz 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28769.log 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.admin.29619.log.gz 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28417.log 2024-09-15T00:12:12.563 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29723.log: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.29723.log.gz 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28381.log 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28769.log.gz 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27809.log 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28417.log.gz 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27748.log 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28381.log: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28381.log.gz 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29023.log 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27809.log.gz 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28592.log 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr: 77.1% -- replaced with /var/log/ceph/ceph-client.admin.27748.log.gz 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28488.log 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.29023.log.gz 2024-09-15T00:12:12.564 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29093.log 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28592.log.gz 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27888.log 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28488.log.gz 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28558.log 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29093.log: 55.6% -- replaced with /var/log/ceph/ceph-client.admin.29093.log.gz 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29793.log 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.27888.log.gz 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28286.log 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28558.log.gz 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28426.log 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29793.log: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.29793.log.gz 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29854.log 2024-09-15T00:12:12.565 INFO:teuthology.orchestra.run.smithi092.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.28286.log.gz 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29820.log 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.28426.log.gz 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27985.log 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29854.log.gz 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29190.log 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29820.log.gz 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28451.log 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27985.log.gz 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29890.log 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29190.log.gz 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28909.log 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28451.log.gz 2024-09-15T00:12:12.566 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28567.log 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29890.log.gz 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29364.log 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28909.log.gz 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27845.log 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr: 56.7% -- replaced with /var/log/ceph/ceph-client.admin.28567.log.gz 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28460.log 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29364.log.gz 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28848.log 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27845.log.gz 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28390.log 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28460.log.gz 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28662.log 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.28848.log.gz 2024-09-15T00:12:12.567 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28628.log 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.28390.log.gz 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27949.log 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28662.log.gz 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27565.log 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28628.log.gz 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.27949.log: -- /var/log/ceph/ceph-client.admin.28089.log 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27949.log.gz 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28136.log 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27565.log.gz 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29199.log 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28089.log.gz 2024-09-15T00:12:12.568 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27017.log 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28136.log.gz 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28989.log 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29199.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.29199.log.gz 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29644.log 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27017.log.gz 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27879.log 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr: 53.1% -- replaced with /var/log/ceph/ceph-client.admin.28989.log.gz 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28980.log 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29644.log.gz 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27705.log 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27879.log.gz 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27257.log 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28980.log.gz 2024-09-15T00:12:12.569 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27434.log 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27705.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27705.log.gz 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29680.log 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr: 57.3% -- replaced with /var/log/ceph/ceph-client.admin.27257.log.gz 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28019.log 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.27434.log.gz 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/ceph-client.admin.29680.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28601.log 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29680.log.gz 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29373.log 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28019.log.gz 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27994.log 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.28601.log.gz 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28742.log 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.29373.log.gz 2024-09-15T00:12:12.570 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27363.log 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.27994.log.gz 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28733.log 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr: 57.2% -- replaced with /var/log/ceph/ceph-client.admin.28742.log.gz 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29653.log 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.27363.log.gz 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27223.log 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28733.log.gz 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28952.log 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.29653.log.gz 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29084.log 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.27223.log.gz 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29443.log 2024-09-15T00:12:12.571 INFO:teuthology.orchestra.run.smithi092.stderr: 76.9% -- replaced with /var/log/ceph/ceph-client.admin.28952.log.gz 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27495.log 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29084.log.gz 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28241.log 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.29443.log.gz 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27186.log 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27495.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27495.log.gz 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29479.log 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28241.log.gz 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28671.log 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27186.log: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.27186.log.gz 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28320.log 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.29479.log.gz 2024-09-15T00:12:12.572 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28207.log 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr: 34.7% -- replaced with /var/log/ceph/ceph-client.admin.28671.log.gz 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27177.log 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr: 54.6% -- replaced with /var/log/ceph/ceph-client.admin.28320.log.gz 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29059.log 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28207.log.gz 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29330.log 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27177.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27177.log.gz 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27248.log 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.29059.log.gz 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27108.log 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29330.log.gz 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29129.log 2024-09-15T00:12:12.573 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27248.log.gz 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29899.log 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27108.log.gz 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29400.log 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29784.log 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.29899.log.gz 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29714.log 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29400.log.gz 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.admin.29129.log.gz 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29233.log 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29784.log.gz 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27599.log 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29714.log.gz 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27678.log 2024-09-15T00:12:12.574 INFO:teuthology.orchestra.run.smithi092.stderr: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.29233.log.gz 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27958.log 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27599.log.gz 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27924.log 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27678.log: 79.0% -- replaced with /var/log/ceph/ceph-client.admin.27678.log.gz 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27284.log 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.27958.log.gz 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28699.log 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.27924.log.gz 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27818.log 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27284.log.gz 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28064.log 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28699.log.gz 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29610.log 2024-09-15T00:12:12.575 INFO:teuthology.orchestra.run.smithi092.stderr: 57.5% -- replaced with /var/log/ceph/ceph-client.admin.27818.log.gz 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28531.log 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28064.log: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.28064.log.gz 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29583.log 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29610.log.gz 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28028.log 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.28531.log.gz 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29409.log 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.29583.log.gz 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27574.log 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.28028.log.gz 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29470.log 2024-09-15T00:12:12.576 INFO:teuthology.orchestra.run.smithi092.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.29409.log.gz 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27608.log 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27574.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.27574.log.gz 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28145.log 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29470.log.gz 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27644.log 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.27608.log.gz 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27538.log/var/log/ceph/ceph-client.admin.28145.log: 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.28145.log.gz 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28311.log 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.27644.log.gz 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27775.log 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.27538.log.gz 2024-09-15T00:12:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.28311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29303.log 2024-09-15T00:12:12.578 INFO:teuthology.orchestra.run.smithi092.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28311.log.gz 2024-09-15T00:12:12.578 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27775.log: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27775.log.gz 2024-09-15T00:12:12.578 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.29303.log: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.29303.log.gz 2024-09-15T00:12:12.578 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-15T00:12:12.578 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m0.057s 2024-09-15T00:12:12.578 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m0.068s 2024-09-15T00:12:12.578 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.074s 2024-09-15T00:12:12.580 INFO:teuthology.orchestra.run.smithi079.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T00:12:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55657.log 2024-09-15T00:12:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41341.log 2024-09-15T00:12:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53596.log 2024-09-15T00:12:12.628 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55657.log.gz 2024-09-15T00:12:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.41341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54353.log 2024-09-15T00:12:12.628 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41341.log.gz 2024-09-15T00:12:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59554.log 2024-09-15T00:12:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53596.log.gz 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57553.log 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54353.log.gz 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59764.log 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59554.log.gz 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53928.log 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57553.log.gz 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57916.log 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59764.log.gz 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53928.log.gz 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41767.log 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58294.log 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57916.log.gz 2024-09-15T00:12:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.41767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46622.log 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41767.log.gz 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57028.log 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58294.log.gz 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37442.log 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46622.log.gz 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50349.log 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57028.log.gz 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54523.log 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37442.log.gz 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45769.log 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50349.log.gz 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36766.log 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54523.log.gz 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54564.log 2024-09-15T00:12:12.630 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45769.log.gz 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36766.log.gz 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59176.log 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59064.log 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54564.log.gz 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37718.log 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59176.log.gz 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31867.log 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59064.log.gz 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60016.log 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37718.log.gz 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53639.log 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31867.log.gz 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36016.log 2024-09-15T00:12:12.631 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60016.log.gz 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53639.log.gz 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35257.log 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58000.log 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36016.log.gz 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53222.log 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35257.log.gz 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58000.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58000.log.gz 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53222.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.48430.log 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53222.log.gz 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.50284.log 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41594.log 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/objectstore_tool.48430.log: /var/log/ceph/objectstore_tool.50284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57070.log 2024-09-15T00:12:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.41594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35313.log 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41594.log.gz 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58336.log 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57070.log.gz 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34688.log 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35313.log.gz 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58748.log 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58336.log.gz 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47221.log 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34688.log.gz 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37635.log 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58748.log.gz 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-09-15T00:12:12.633 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47221.log.gz 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39084.log 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37635.log.gz 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58517.log 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48347.log 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39084.log.gz 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40044.log 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58517.log.gz 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35042.log 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48347.log.gz 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47667.log 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40044.log.gz 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47302.log 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35042.log.gz 2024-09-15T00:12:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51338.log 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% 95.1% -- replaced with /var/log/ceph/objectstore_tool.48430.log.gz 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47667.log.gz 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38681.log 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47302.log.gz 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59806.log 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51338.log.gz 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.50284.log.gz 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38681.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.38681.log.gz /var/log/ceph/ceph-client.admin.45852.log 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35045.log 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59806.log.gz 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.45852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50409.log 2024-09-15T00:12:12.635 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45852.log.gz 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37760.log 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35045.log.gz 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.50409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50409.log.gz 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40319.log 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53089.log 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37760.log.gz 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40319.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.40319.log.gz /var/log/ceph/ceph-client.admin.60056.log 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35450.log 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53089.log.gz 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.60056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36398.log 2024-09-15T00:12:12.636 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60056.log.gz 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52587.log 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35450.log.gz 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57449.log 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36398.log.gz 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52587.log: /var/log/ceph/ceph-client.admin.57533.log 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52587.log.gz 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57449.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.57449.log.gz --verbose 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/objectstore_tool.42865.log 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59218.log 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57533.log.gz 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/objectstore_tool.42865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58699.log 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59218.log.gz 2024-09-15T00:12:12.637 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59302.log 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47780.log 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58699.log.gz 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41665.log 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59302.log.gz 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47728.log 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47780.log.gz 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.41665.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.59033.log 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57637.log 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47728.log.gz 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59033.log.gz 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39335.log 2024-09-15T00:12:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57637.log.gz 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39335.log.gz 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35041.log 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53849.log 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35041.log.gz 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35315.log 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59974.log 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53849.log.gz 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35315.log.gz 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52797.log 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34820.log 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59974.log.gz 2024-09-15T00:12:12.639 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52797.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52797.log.gz 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.38077.log 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46451.log 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34820.log: 95.1% -- replaced with /var/log/ceph/objectstore_tool.42865.log.gz 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34820.log.gz 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38291.log 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38077.log.gz 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55332.log 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46451.log.gz 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56235.log 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38291.log.gz 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55332.log.gz 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35258.log 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35470.log 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56235.log.gz 2024-09-15T00:12:12.640 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38997.log 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35258.log.gz 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35470.log.gz 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35610.log 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40731.log 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38997.log.gz 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35610.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.35610.log.gz -- 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph.audit.log 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46835.log 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40731.log.gz 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59470.log 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46835.log.gz 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38359.log 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58084.log 2024-09-15T00:12:12.641 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59470.log: 28.9% -- replaced with /var/log/ceph/ceph-client.admin.59470.log.gz 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38359.log.gz 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55220.log 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58084.log.gz 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55220.log.gz 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43346.log 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58944.log 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43346.log.gz 2024-09-15T00:12:12.642 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48115.log 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31566.log 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58944.log.gz 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48115.log.gz 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57958.log 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58749.log 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr: 28.1% -- replaced with /var/log/ceph/ceph-client.admin.31566.log.gz 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57958.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57958.log.gz 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.49352.log 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58126.log 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58749.log.gz 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/objectstore_tool.49352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59722.log 2024-09-15T00:12:12.643 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58126.log.gz 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37634.log 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58510.log 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59722.log.gz 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41168.log 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37634.log.gz 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59512.log 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58510.log.gz 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.41168.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41852.log 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41168.log.gz 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53139.log 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59512.log.gz 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/objectstore_tool.41852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35243.log 2024-09-15T00:12:12.644 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35912.log 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53139.log.gz 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35513.log 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35243.log.gz 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54836.log 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35912.log.gz 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35039.log 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35513.log.gz 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36136.log 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54836.log.gz 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35039.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27183.log 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35039.log.gz 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36136.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36136.log.gz 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58519.log 2024-09-15T00:12:12.645 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.tmp-client.admin.27183.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27183.log.gz 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47993.log 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.49352.log.gz 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34862.log 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58519.log.gz 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47993.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47993.log.gz 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47942.log 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34862.log.gz 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59680.log 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47942.log.gz 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58518.log 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57324.log 2024-09-15T00:12:12.646 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59680.log.gz 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55119.log 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58518.log.gz 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53304.log 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57324.log.gz 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.41852.log.gz 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35702.log 0.0% 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55119.log.gz 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57325.log 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53304.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53304.log.gz 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58773.log 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35702.log.gz 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38035.log 2024-09-15T00:12:12.647 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57325.log.gz 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35198.log 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58773.log.gz 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57874.log 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38035.log.gz 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35198.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.35198.log.gz 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37802.log 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59428.log 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57874.log.gz 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39037.log 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37802.log.gz 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59386.log 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59428.log.gz 2024-09-15T00:12:12.648 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41401.log 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39037.log.gz 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35744.log 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59386.log.gz 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.41401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41401.log.gz 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57617.log 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55443.log 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35744.log.gz 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47099.log 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57617.log.gz 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40594.log 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55443.log.gz 2024-09-15T00:12:12.649 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46237.log 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47099.log.gz 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40594.log.gz 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38934.log 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46237.log.gz 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59890.log 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38934.log.gz 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43259.log 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58252.log 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59890.log.gz 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.43259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58210.log 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43259.log.gz 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59260.log 2024-09-15T00:12:12.650 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58252.log.gz 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56813.log 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58210.log.gz 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36673.log 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59260.log.gz 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56813.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56813.log.gz 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58378.log 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36673.log.gz 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35954.log 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58378.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58378.log.gz 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53160.log 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35954.log.gz 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35828.log 2024-09-15T00:12:12.651 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53160.log.gz 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57832.log 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35828.log.gz 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35517.log 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57832.log.gz 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.43838.log 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35517.log.gz 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42751.log 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/objectstore_tool.43838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58752.log 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42751.log.gz 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58752.log.gz 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39806.log 2024-09-15T00:12:12.652 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59596.log 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39806.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34769.log 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39806.log.gz 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51428.log 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59596.log.gz 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41006.log 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34769.log.gz 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51428.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51428.log.gz 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.42285.log 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35786.log 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.42285.log: /var/log/ceph/ceph-client.admin.41006.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42285.log.gz 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41006.log.gz 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47516.log 2024-09-15T00:12:12.653 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35786.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56875.log 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35786.log.gz 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47516.log.gz 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47048.log 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.56875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56875.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44312.log 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.43838.log.gz 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58966.log 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47048.log: /var/log/ceph/ceph-client.admin.44312.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.47048.log.gz 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34744.log 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44312.log.gz 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36139.log 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58966.log: /var/log/ceph/ceph-client.admin.34744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58924.log 2024-09-15T00:12:12.654 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58966.log.gz 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34744.log.gz 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36139.log.gz 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.44810.log 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.58924.log: -- /var/log/ceph/ceph-client.admin.46875.log 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58924.log.gz 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/objectstore_tool.44810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37227.log 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46875.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40339.log 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37227.log.gz 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46024.log 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40339.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T00:12:12.655 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38506.log 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.46024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46024.log.gz 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40085.log 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38506.log.gz 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40085.log.gz 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39902.log 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31795.log 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58508.log 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39902.log.gz 2024-09-15T00:12:12.656 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31795.log.gz 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34904.log 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58508.log.gz 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.44810.log.gz 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46664.log 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34904.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34904.log.gz 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57742.log 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46664.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46664.log.gz 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58520.log 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57742.log.gz 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51619.log 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58520.log.gz 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39334.log 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.51619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51619.log.gz 2024-09-15T00:12:12.657 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57491.log 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39334.log.gz 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40912.log 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57491.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57491.log.gz 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39704.log 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40912.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57700.log 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40912.log.gz 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45983.log 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39704.log.gz 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46196.log 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57700.log: /var/log/ceph/ceph-client.admin.45983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58042.log 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57700.log.gz 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45983.log.gz 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.46196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46196.log.gz 2024-09-15T00:12:12.658 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47394.log 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.58042.log: -- /var/log/ceph/ceph-client.admin.49565.log 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58042.log.gz 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.47394.log: 28.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53427.log 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47394.log.gz 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.49565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39570.log 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49565.log.gz 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36137.log 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.53427.log: /var/log/ceph/ceph-client.admin.39570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54988.log 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53427.log.gz 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39570.log.gz 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36137.log.gz 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36970.log 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38914.log 2024-09-15T00:12:12.659 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54988.log: /var/log/ceph/ceph-client.admin.36970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44771.log 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54988.log.gz 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36970.log.gz 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38914.log.gz 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35552.log 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55749.log 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.44771.log: gzip/var/log/ceph/ceph-client.admin.35552.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35552.log.gz 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.52121.log 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.55749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44771.log.gz 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55749.log.gz 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38639.log 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52121.log.gz 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35870.log 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38639.log.gz 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36868.log 2024-09-15T00:12:12.660 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35870.log.gz 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37072.log 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36868.log.gz 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57785.log 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37072.log.gz 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52877.log 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57785.log.gz 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54814.log 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.52877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52877.log.gz 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59848.log 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54814.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54814.log.gz 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48206.log 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59848.log.gz 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59932.log 2024-09-15T00:12:12.661 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.48206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48206.log.gz 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39807.log 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59932.log.gz 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39807.log.gz 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59344.log 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40681.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59638.log 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59344.log.gz 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40636.log 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59638.log.gz 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57263.log 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40636.log.gz 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59027.log 2024-09-15T00:12:12.662 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.57263.log: 0.0%gzip -5 --verbose -- /var/log/ceph/objectstore_tool.51588.log 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57263.log.gz 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.59027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35046.log 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59027.log.gz 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58700.log 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/objectstore_tool.51588.log: /var/log/ceph/ceph-client.admin.35046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35043.log 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35046.log.gz 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36950.log 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58700.log.gz/var/log/ceph/ceph-client.admin.35043.log: 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35043.log.gz 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37320.log 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36950.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39643.log 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36950.log.gz 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54191.log 2024-09-15T00:12:12.663 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37320.log: /var/log/ceph/ceph-client.admin.39643.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37320.log.gz 2024-09-15T00:12:12.664 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39643.log.gz 2024-09-15T00:12:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58168.log 2024-09-15T00:12:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54191.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54121.log 2024-09-15T00:12:12.664 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T00:12:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.58168.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58168.log.gz 2024-09-15T00:12:12.664 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.54121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54121.log.gz 2024-09-15T00:12:12.664 INFO:teuthology.orchestra.run.smithi027.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.51588.log.gz 2024-09-15T00:12:12.664 INFO:teuthology.orchestra.run.smithi027.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T00:12:12.737 INFO:teuthology.orchestra.run.smithi027.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T00:12:15.384 INFO:teuthology.orchestra.run.smithi027.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T00:12:16.032 INFO:teuthology.orchestra.run.smithi027.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T00:12:16.048 INFO:teuthology.orchestra.run.smithi079.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T00:12:16.932 INFO:teuthology.orchestra.run.smithi027.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T00:12:16.933 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T00:12:16.934 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m4.506s 2024-09-15T00:12:16.934 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m11.062s 2024-09-15T00:12:16.934 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.536s 2024-09-15T00:12:17.195 INFO:teuthology.orchestra.run.smithi079.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T00:12:18.679 INFO:teuthology.orchestra.run.smithi079.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T00:12:18.681 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-15T00:12:18.681 INFO:teuthology.orchestra.run.smithi079.stderr:real 0m6.253s 2024-09-15T00:12:18.681 INFO:teuthology.orchestra.run.smithi079.stderr:user 0m14.205s 2024-09-15T00:12:18.681 INFO:teuthology.orchestra.run.smithi079.stderr:sys 0m0.589s 2024-09-15T00:12:18.682 INFO:tasks.ceph:Archiving logs... 2024-09-15T00:12:18.682 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/remote/smithi027/log 2024-09-15T00:12:18.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T00:12:20.137 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/remote/smithi079/log 2024-09-15T00:12:20.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T00:12:21.884 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/remote/smithi092/log 2024-09-15T00:12:21.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T00:12:22.015 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T00:12:22.035 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T00:12:22.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T00:12:22.038 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T00:12:22.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T00:12:22.143 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T00:12:22.152 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T00:12:22.152 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:12:22.155 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:12:22.158 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:12:22.171 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:12:22.172 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:12:22.175 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:12:22.175 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:12:22.176 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-15T00:12:22.176 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +254us[ +255us] +/- 53ms 2024-09-15T00:12:22.176 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 +440us[ +441us] +/- 57ms 2024-09-15T00:12:22.176 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -1164us[-1164us] +/- 90ms 2024-09-15T00:12:22.176 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +275us[ +266us] +/- 53ms 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 +439us[ +439us] +/- 57ms 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -1162us[-1162us] +/- 90ms 2024-09-15T00:12:22.178 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +271us[ +266us] +/- 53ms 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 +449us[ +449us] +/- 57ms 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -1154us[-1159us] +/- 90ms 2024-09-15T00:12:22.180 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:12:22.181 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T00:12:22.189 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T00:12:22.189 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T00:12:22.198 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T00:12:22.302 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T00:12:22.381 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T00:12:22.447 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:12:22.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:12:22.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:12:22.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:12:22.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:12:22.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:12:22.541 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T00:12:22.549 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T00:12:22.558 INFO:teuthology.task.internal:Duration was 633.762496 seconds 2024-09-15T00:12:22.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T00:12:22.566 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T00:12:22.566 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T00:12:22.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T00:12:22.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T00:12:22.610 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:12:22.610 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:12:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:12:22.906 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T00:12:22.906 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T00:12:22.907 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T00:12:22.960 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-09-15T00:12:22.960 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T00:12:22.981 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T00:12:22.982 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T00:12:23.002 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T00:12:23.002 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T00:12:23.005 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T00:12:23.025 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T00:12:23.069 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T00:12:23.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T00:12:23.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T00:12:23.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T00:12:23.215 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T00:12:23.226 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T00:12:23.226 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T00:12:23.252 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T00:12:23.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T00:12:23.302 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T00:12:23.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:12:23.322 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:12:23.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:12:23.342 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-09-15T00:12:23.345 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-09-15T00:12:23.365 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-15T00:12:23.389 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:12:23.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:12:23.420 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:12:23.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:12:23.435 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:12:23.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:12:23.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T00:12:23.459 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T00:12:23.460 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/remote/smithi027 2024-09-15T00:12:23.460 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T00:12:23.497 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/remote/smithi079 2024-09-15T00:12:23.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T00:12:23.534 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904812/remote/smithi092 2024-09-15T00:12:23.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T00:12:23.571 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T00:12:23.571 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T00:12:23.574 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T00:12:23.576 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T00:12:23.620 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T00:12:23.631 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T00:12:23.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T00:12:23.641 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T00:12:23.641 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T00:12:23.644 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T00:12:23.646 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T00:12:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 00:12 /home/ubuntu/cephtest 2024-09-15T00:12:23.658 INFO:teuthology.orchestra.run.smithi079.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 00:12 /home/ubuntu/cephtest 2024-09-15T00:12:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 00:12 /home/ubuntu/cephtest 2024-09-15T00:12:23.675 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T00:12:23.686 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T00:12:23.708 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 633.7624955177307 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T00:12:23.708 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T00:12:23.794 INFO:teuthology.run:pass